Most AI demos today can talk really well, but they can't actually do anything useful. In this project, we're going to change that by building a voice-controlled AI assistant using the open-source XiaoZhi AI project and the feature-packed DFRobot ESP32-S3 AI Camera. What results isn't just a chatbot—it's a real, functional AI companion that can see its surroundings, control hardware, and even manage your calendar.
This assistant lets you turn lights on and off, fetch sensor data, take photos on command, and conduct real visual recognition conversations. The best part? The entire project is open source under the MIT license, meaning you can use it for free, even in commercial applications. If you're ready to build an AI that's actually useful, read on.
Get PCBs for Your Projects ManufacturedYou must check out PCBWAY for ordering PCBs online for cheap!
You get 10 good-quality PCBs manufactured and shipped to your doorstep for cheap. You will also get a discount on shipping on your first order. Upload your Gerber files onto PCBWAY to get them manufactured with good quality and quick turnaround time. PCBWay now could provide a complete product solution, from design to enclosure production. Check out their online Gerber viewer function. With reward points, you can get free stuff from their gift shop. Also, check out this useful blog on PCBWay Plugin for KiCad from here. Using this plugin, you can directly order PCBs in just one click after completing your design in KiCad.
What You'll NeedHardware- 1 × DFRobot ESP32-S3 AI Camera – This is the star of the show. It comes with an onboard microphone, speaker, camera, and ample processing power.
- 1 × Type-C USB cable – For power and programming.
- XiaoZhi AI Firmware – The core open-source ESP32 chatbot project.
- ESP-IDF or pre-built firmware – Your choice of building from source or a quick flash.
- XiaoZhi Control Panel – For device management and customization (available at xiaozhi.me).
Before we dive in, let's appreciate what the DFRobot ESP32-S3 AI Camera brings to the table. Powered by the Espressif ESP32-S3 chip, it's specifically designed for smart home and IoT applications. Here's the full breakdown:
- Main Processor: Xtensa® 32-bit LX7 dual-core processor running at up to 240MHz, paired with 16MB Flash and 8MB PSRAM.
- Camera Module (OV3660): 2MP sensor with a 160° wide-angle lens, capable of both visible light and 940nm infrared sensing—that means it works just as well in total darkness as it does in broad daylight.
- Audio System: Integrated I2S PDM microphone for capturing your voice, plus a MAX98357 I2S amplifier chip and speaker connector for playback.
- Additional On-Board Perks: LTR‑308 ambient light sensor, reset and boot buttons, SD card slot for storage expansion, and a GPIO‑controlled LED for status indications.
You have two options for flashing the XiaoZhi firmware: a quick and easy path using a pre‑compiled binary, or a build‑from‑source path using ESP‑IDF. If you're just getting started, I recommend the pre‑built method—it's much faster and less error‑prone.
Step 1: Flash the XiaoZhi FirmwareLet's get the firmware onto your ESP32-S3 AI Camera. Visit the following git repo and you can find the xiaozhi esp32 s3 ai camera bin.
Put your board into download mode. Press and hold the BOOT button while connecting the USB cable, then release the button. Your computer should detect the board.
Launch the Flash Download Tool. In the tool:
- Set Chip Type:
ESP32‑S3 - Set WorkMode:
Develop - Set LoadMode:
UART
- Load the firmware file. In the first blank row, click the
...button and select the.binfirmware file you downloaded. Set the start address to0x00.
- Select the correct COM port. Choose the port your board is connected to and set the baud rate to 1152000.
- Erase and flash. Click ERASE to clear any existing firmware, then click START to flash the XiaoZhi firmware onto the board.
After a successful flash, press the RST button on your ESP32-S3 AI Camera. The device will boot into Wi-Fi configuration mode, broadcasting a hotspot named something like Xiaozhi-xxxx.
- Connect to the hotspot using your phone or computer.
- A configuration page will automatically appear. Select your local 2.4GHz Wi‑Fi network (5GHz networks are not supported) and enter the password.
- Wait for connection. Once connected, the device will generate a unique device code displayed on the terminal screen
Don't lose this device code—you'll need it for activation in the next step!Step 3: Activate Your Device via the XiaoZhi Control Panel
Now it's time to officially bring your assistant online.
- Head over to https://xiaozhi.meand create a free account.
- Navigate to the Console (Dashboard)
- Click "Add Device" or the equivalent activation button.
- Enter the device code from your ESP32-S3's terminal (or the voice announcement).
- Then you can customize the agent based on your need.
- Restart your device. Once it reconnects, your AI assistant is officially live and ready to talk
Your assistant is now up and running. You'll likely see a friendly face or animated expression on the display, letting you know it's ready.
Wake-Up MethodsYou have multiple ways to wake your assistant from its idle state:
- Voice wake‑up: Say "Hello XiaoZhi" or "Hi ESP" to grab its attention.
- Physical button: Press the BOOT button.
Once the assistant acknowledges the wake call, you can begin speaking your request.
What You Can DoWith the XiaoZhi firmware on your DFRobot ESP32-S3, you can ask:
- General questions: "What's the weather like today?"
- Timers and reminders: "Set a timer for 10 minutes."
- Hardware control: "Turn on the LED." (The MCP protocol bridges AI decisions to physical action).
- Photography: "Take a photo for me." (We'll enable this in the next step).
Congratulations! You've just built a smart, voice‑controlled AI assistant using the DFRobot ESP32-S3 AI Camera and the open‑source XiaoZhi AI project. What you now hold in your hands is more than just a talking gadget—it's a fully functional AI companion that can see, speak, and even control the world around it.
The open‑source nature of XiaoZhi means the possibilities are truly endless. Whether you're building a smart home controller, a visual assistant for the kitchen, or just a fun new toy to experiment with, the only real limit is your imagination.
Have fun building, and don't forget to share your creation with the community!
Comments