ESP32-S3 Camera Object Detection

Edge AI puts machine learning directly on small, low-power devices — no round trip to the cloud, no ongoing subscription, and no dependency on an internet connection to make a decision. In this project we turn a NORVI ESP32-S3 Camera module into a self-contained object detection system: it captures its own training images over Wi-Fi, we train an image classification model in Edge Impulse, and we deploy that model straight back onto the ESP32-S3 so it can recognize objects in real time from its own video stream.

The workflow has four stages: set up the Arduino development environment for the ESP32-S3, run the camera as a web server to collect a labeled image dataset, train and export a TinyML model in Edge Impulse, and finally flash the exported model back onto the board for on-device inference. Every stage below is illustrated with the exact screens and code used in the build.

Step 1: Set Up the Development Environment

1.1 Install the Arduino IDE

Go to the official Arduino website (arduino.cc) and download the Arduino IDE for your operating system. Install the tool and launch it from the desktop icon.

1.2 Add the ESP32-S3 Board via Board Manager

Open File → Preferences to bring up the Preferences dialog, then find the field labeled “Additional Boards Manager URLs.” If it already contains a URL, add a comma after it, then paste in the ESP32 package index URL below and click OK to save:

https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json

Next, open the Boards Manager (Tools → Board → Boards Manager), search for “esp32, ” and install the package. The IDE downloads and installs the corresponding board definitions automatically.

Once installation finishes, open Tools → Board and select “ESP32S3 Dev Module” under the ESP32 Arduino section.

Configure the remaining chip-specific settings (PSRAM, partition scheme, flash size, etc.) to match the ESP32-S3 camera module as shown below.

Step 2: Set Up the NORVI ESP32-S3 Camera Web Server

Download the Camera Web Server sketch for the OV5640 auto-focus camera from the NORVI GitHub repository:

https://github.com/NORVIControllers/NORVI-ESP32-CAMERA/tree/main/CameraWebServer_OV5640AF

2.1 Install the Camera Library

In the Arduino Library Manager, search for “OV5640 Auto Focus for ESP32 Camera” and click Install.

2.2 Install the Wi-Fi Library

In the same Library Manager, search for “WiFi” and install the library used to connect the camera to your network.

2.3 Understanding the Autofocus Code

The sketch initializes the camera sensor and enables the OV5640's autofocus routine before the camera starts streaming:

sensor_t* sensor = esp_camera_sensor_get();
// Gets the sensor object needed to configure the OV5640 module
 
ov5640.start(sensor);
// Initializes the OV5640 camera with the sensor settings
 
if (ov5640.focusInit() == 0) {
  Serial.println("OV5640_Focus_Init Successful!");
}
// Initializes autofocus; prints a status message on success
 
if (ov5640.autoFocusMode() == 0) {
  Serial.println("OV5640_Auto_Focus Successful!");
}
// Enables continuous autofocus mode

2.4 Configure and Upload

Enter your Wi-Fi credentials (SSID and password) in the sketch before uploading:

Connect the ESP32 camera to your computer with a USB-C data cable, then in the Arduino IDE select the correct serial port under Tools → Port. Compile the sketch and click Upload to flash it to the NORVI ESP32-S3 Camera.

Open the Serial Monitor after the upload completes — the board prints the IP address of its built-in web server.

Navigate to that IP address in a web browser to view the live camera stream. Use the Capture button to save still images to your computer — this is the dataset you will use to train the object detection model in the next step.

Step 3: Train the Model with Edge Impulse

3.1 Create an Edge Impulse Project

Edge Impulse is a leading development platform for machine learning on edge devices. Sign in with your account, create a new project, and select “Espressif ESP-EYE (ESP32)” as the target device.

3.2 Upload Data for Training

Open the Data Acquisition tab and upload the images captured from the camera web server.

Label each image according to the object it contains, then save the labels.

3.3 Design and Train the Impulse

In the Impulse Design tab, create an impulse using an image data block and an image classification (transfer learning) block. Starting from the raw images, the pipeline resizes each one to 96×96 pixels before feeding it into the transfer learning block.

Configure the processing and learning blocks, then save the impulse.

3.4 Pre-Processing (Feature Generation)

Besides resizing, convert the images to grayscale rather than keeping full RGB color depth. With grayscale, each sample has a dimension of 9, 216 features (96×96×1)-roughly a third of the size of an RGB sample-which reduces the memory required for on-device inference.

Click Save Parameters, then Generate Features.

Before training, review and adjust the training parameters (epochs, learning rate, validation split, etc.) shown below.

Start training the model on the uploaded, labeled images.

3.5 Deploy the Model to the ESP32-S3

Once training is complete, open the Deployment tab, select ESP32 as the target, and export the trained model as an Arduino library.

Step 4: Modify the ESP32-S3 Code to Use the Model

Download the generated library.zip and add it to the Arduino IDE via Sketch → Include Library → Add.ZIP Library. An example sketch for your project will then appear under File → Examples.

Update the camera model define, pin configuration, and autofocus initialization to match the OV5640 module. The key integration points are shown below.

#include "ESP32_OV5640_AF.h"
 
#define CAMERA_MODEL_ESP32S3_CAM_LCD
#elif defined(CAMERA_MODEL_ESP32S3_CAM_LCD)
#define PWDN_GPIO_NUM   41  // POWER
#define RESET_GPIO_NUM  42
#define XCLK_GPIO_NUM   15  // MCLK
#define SIOD_GPIO_NUM   4   // SDA
#define SIOC_GPIO_NUM   5   // SCL
#define Y9_GPIO_NUM     16
#define Y8_GPIO_NUM     17
#define Y7_GPIO_NUM     18
#define Y6_GPIO_NUM     12
#define Y5_GPIO_NUM     10
#define Y4_GPIO_NUM     8
#define Y3_GPIO_NUM     9
#define Y2_GPIO_NUM     11
#define VSYNC_GPIO_NUM  6
#define HREF_GPIO_NUM   7
#define PCLK_GPIO_NUM   13
#define LED_GPIO_NUM    14
#else
#error "Camera model not selected"
#endif
 
OV5640 ov5640 = OV5640();
 
void setup() {
  Serial.begin(115200);
 
  sensor_t* sensor = esp_camera_sensor_get();
  ov5640.start(sensor);
 
  if (ov5640.focusInit() == 0) {
    Serial.println("OV5640_Focus_Init Successful!");
  }
  if (ov5640.autoFocusMode() == 0) {
    Serial.println("OV5640_Auto_Focus Successful!");
  }
 
  if (sensor->id.PID == OV3660_PID) {
    sensor->set_vflip(sensor, 1);       // flip it back
    sensor->set_brightness(sensor, 1);  // up the brightness a bit
    sensor->set_saturation(sensor, -2); // lower the saturation
  }
}

Compile and upload the sketch to the NORVI ESP32-S3 Camera. Once it boots, it will start classifying objects using the model you trained.

Step 5: Test the Model (Inference)

With the sketch running, the ESP32-S3 continuously processes its live video stream through the deployed Edge Impulse model. Classification results are printed to the Serial Monitor in real time as objects appear in the camera's field of view.

Conclusion

This project takes an ESP32-S3 camera from a blank board to a working, on-device object detector: the camera itself was used to build the training dataset, Edge Impulse handled feature extraction and model training, and the exported model runs entirely on the microcontroller with no cloud inference required. From here, the same workflow can be extended to more object classes or higher-resolution features or paired with actuators (relays, servos, displays) to trigger actions when specific objects are recognized.