I Built a Standalone Edge AI Tracking Camera

Introduction

This is a small object-tracking camera that can detect an object and turn to follow it.

My first version used an ESP32-CAM to stream video to my laptop, where a YOLO model detected the object. The laptop then sent movement commands to another ESP32 controlling the servos.

It worked, but the system still depended on my laptop.

In this upgraded version, object detection runs locally on Grove Vision AI V2, while a XIAO ESP32-S3 controls the pan and tilt servos.

I also redesigned the 3D-printed body to fit the new hardware better.

How It Works

The tracking loop :

1. Grove Vision AI V2 detects the object.

2. The detected object position is sent to the XIAO.

3. The XIAO compares the object position with the center of the camera frame.

4. If the object is off-center, the XIAO moves the pan and tilt servos.

5. The camera turns toward the object.

Object detection and tracking now run locally on the device.

Hardware Used

For this prototype, I used:

3D-Printed Camera Structure

I redesigned the camera body to fit the Grove Vision AI V2 hardware, the OV5647 camera module, the XIAO board, and the MG90S servos.

Servo Control

The XIAO moves the camera left, right, up, or down based on where the object appears in the camera frame.

First, it checks how far the object is from the center:

errorX = objectCenterX - frameCenterX;
errorY = objectCenterY - frameCenterY;

errorX shows how far the object is from the center horizontally.

errorY shows how far the object is from the center vertically.

If the object is far from the center, the camera moves faster. If it is only slightly off-center, the camera moves more slowly. Instead of jumping directly to a new servo position, the XIAO calculates a target velocity for each axis and gradually adjusts the servo speed.

In simplified form, the target servo speed is calculated like this:

panSpeed  = clamp(KpPan * errorX, -maxPanSpeed, maxPanSpeed);
tiltSpeed = clamp(KpTilt * errorY, -maxTiltSpeed, maxTiltSpeed);

A small dead zone stops the camera from making unnecessary movements when the object is already close to the center.

The detected object position is also filtered before it is used for servo control. This reduces jitter caused by small changes in the detection box.

The servo velocity changes gradually instead of jumping instantly, which helps make the movement smoother and more stable.

Accurate detection alone is not enough. The detected position also needs to be filtered, and the servo movement needs to remain smooth for reliable tracking.

If the object is no longer detected, the system briefly waits for a fresh detection and then stops the servo movement until the target appears again.

AI inference and servo control run as separate tasks, allowing the servos to update independently while new detections are being processed.

Demo Results

The upgraded prototype is now able to track an object locally using Grove Vision AI V2 and XIAO.

The most important improvement is that the object detection no longer depends on my laptop.

Now the Grove Vision AI V2 detects the object directly on the device. It sends the object position to the XIAO, and the XIAO moves the pan and tilt servos to follow it.

This is not a finished product yet, but the main local edge AI tracking loop is working.

Original Version vs Upgraded Version

Camera system

Original version: ESP32-CAM

Upgraded version: Grove Vision AI V2 + OV5647-62 FOV camera

AI inference

Original version: Laptop

Upgraded version: Grove Vision AI V2

Servo control

Original version: Second ESP32 + PCA9685 servo driver

Upgraded version: XIAO ESP32-S3 + PCA9685 servo driver

Servos

Original version: Larger MG995/MG996-style servos

Upgraded version: MG90S metal gear micro servos

Tracking loop

Original version: Laptop YOLO + serial commands

Upgraded version: Local detection + XIAO servo control

Portability

Original version: Limited

Upgraded version: More compact

Main limitation

Original version: Required laptop inference

Upgraded version: Still needs a cleaner enclosure and power management

What Still Needs Improvement

This project is still a prototype, so there are several things I want to improve next:

• Cleaner wiring
• A better enclosure for the electronics
• Battery power and power management
• Smoother servo movement
• Better mechanical balance
• More latency and response-time testing
• The current AI model was trained on a dataset of about 500 images. I believe the detection accuracy and reliability could be improved with a larger and more diverse training dataset.

One future idea is to turn this into a nature or wildlife monitoring camera. For that, the system would need a more complete enclosure, reliable power, and environmental protection.

Conclusion

This project started as an upgrade to my original ESP32-CAM object tracking camera.

There were a lot of small problems to solve along the way, but seeing it finally detect an object and move to follow it made the whole process worth it.

It is still a prototype, but now it feels much closer to the device I originally imagined.