Quick Start with RPi AI Camera and Pre-trained AI Models

What is Raspberry Pi AI Camera?

The Raspberry Pi AI Camera is a camera module that connects to the Raspberry Pi series via the Camera Serial Interface (CSI) on the board, similar to the previous Raspberry Pi Cameras. The key difference is that this module can perform AI inference directly within the camera sensor.

At the heart of this module is Sony’s IMX500 Intelligent Sensor, which features built-in logic circuits capable of running neural network inference. This means the camera can output not only image data but also Computer Vision AI inference results directly from the module. This represents a significant advancement over traditional camera sensors, enabling edge AI applications without the need for external AI capable hardware such as GPU or powerful CPU to perform AI inference. If you’d like to learn more about the IMX500, click here.

Goal for This Project

This project offers a step-by-step quick-start guide for those who have just started using the new Raspberry Pi AI Camera. It helps you quickly explore its built-in AI inference capabilities.

What You'll Need

• Raspberry Pi 5 (RPi 5)
• Raspberry Pi AI Camera (IMX500)
• microSD card (32GB+ recommended)
• Internet connection to setup and update IMX500 libraries.
• Keyboard, mouse, monitor (for local setup)
• Windows/Ubuntu/Mac PC for flushing Raspbian OS

Step-by-step Instructions

Hardware Setup

1️⃣ Connect the Camera to the Raspberry Pi 5

Securely connect your Raspberry Pi AI Camera to the CSI port on your RPi 5. Please refer to the video below for Hardware setup.

Environment Setup

2️⃣ Flash Raspbian OS to SD card

Download Raspberry Pi imager to your PC and install it. The Raspberry Pi Imager is a tool that helps you write Raspbian OS image to a microSD card for booting your Raspberry Pi.

Launch the application by clicking the Raspberry Pi Imager icon.

Click Choose device and select Raspberry Pi 5 from the list.

Next, click CHOOSE OS and select Raspberry Pi OS (64-bit) from the list. Imager always shows the recommended version of Raspberry Pi OS at the top of the list.

Plug a microSD card in using an external or built-in SD card reader. Then, click Choose storage and select your storage device.

When you click Next to proceed, Imager will ask you to apply OS customization. It is strongly recommended to configure your Raspberry Pi via the OS customization settings. Click the Edit Settings button to open OS customization.

In OS customization, You can preconfigure:

• Username and password
• Wi-Fi connection setting
• Device hostname
• Time zone
• Keyboard layout
• Locale (or language settings)
• Remote connectivity

After saving OS customization, you will see the popup showing the confirmation for writing OS image. Click Yes when you are ready to go.

When you see the "Write Successful" popup, your image has been completely written and verified. You’re now ready to boot a Raspberry Pi from the storage device! Click Continue on the popup, remove the microSD card, then insert the microSD card into your Raspberry Pi 5.

3️⃣ Update to the Latest Raspbian OS which contains IMX500 software

Make sure that the microSD card you wrote OS onto is inserted your Raspberry Pi 5 and power it on.

After booting and logging in Raspberry Pi OS, open a terminal and ensure that your Raspberry Pi runs the latest software. Run the following command to update:

sudo apt update && sudo apt -y full-upgrade
sudo apt install -y imx500-all

imx500-all package makes it easy to get started with the IMX500 sensor on Raspberry Pi. It bundles all the necessary drivers and tools so you can run AI models right on the camera.

Once update is complete, ensure to reboot the system by running:

sudo reboot now

Please refer to Raspberry Pi Getting Started documentation for more details.

Sample Code Preparation

4️⃣ Get sample codes for PiCamera2

Picamera2 is a powerful and easy-to-use Python library for working with Raspberry Pi camera modules. It lets you capture images, record video, and even integrate with AI tools like OpenCV.

Picamera2 offers sample codes for pre-trained AI models such as image classification, object detection, object segmentation, and pose estimation, see the picamera2 GitHub repository.

Most of the examples use OpenCV for some additional processing. To install the dependencies required to run OpenCV, run the following command:

sudo apt install -y python3-opencv python3-munkres

To get sample codes, clone the repository by running the following command.

git clone https://github.com/raspberrypi/picamera2.git
cd picamera2/examples/imx500

Run the Sample Demo

5️⃣ Try pre-trained AI models

With the pre-trained models bundled in imx500-all, you can skip the training step and jump straight into testing AI inference. Just run the sample code to see it in action.5-1. YOLOv8Run the following script from the repository to run YOLOv8 object detection:

python imx500_object_detection_demo.py --model /usr/share/imx500-models/imx500_network_ssd_mobilenetv2_fpnlite_320x320_pp.rpk

You can see the inference result visually like this.

5-2. Pose EstimationTo try pose estimation in Picamera2, run the following script from the repository:

python imx500_pose_estimation_higherhrnet_demo.py

And here is the result you can see.

What's Next?

The Raspberry Pi AI Camera also allows you to deploy your own custom AI models. To do this, you'll need to convert your model into a compatible format before deployment.

If you're interested in learning more about the model conversion process, click here.