Yolov8 on Raspberry PI5 with Coral TPU

Installing yolov8 on RPI5 is very simple:

sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get autoremove -y
python3 -m venv yolo_env
source yolo_env/bin/activate
pip3 install ultralytics

There is also a shell scrip available for download, which includes the same lines above and the additional download of the GardenCam videos and models plus test runs:

wget https://github.com/StefansAI/Yolov8_Rpi5_CoralUSB/raw/main/scripts/yolov8_install.sh
chmod +x yolov8_install.sh
./yolov8_install.sh

=======================================================================

To run the Coral TPU with the Raspberry Pi 5 I had to research a lot, since nothing was straight forward. One reason is, that Google stopped supporting their software support for their TPU long time ago. But Python has evolved and the old Google installations don't work anymore.

I mainly followed this article:Coral Edge TPU on a Raspberry Pi with Ultralytics YOLOv8, but also JungLearnBot/RPi5_yolov8/Coral_TPU and of course Get started with the USB Accelerator.

In addition I installed the edge_tpu_silva version in a python 3.9 venv for comparison.

It took me some time, where I could not get any exported model to run on the TPU at all. Finally, after some debugging, I had to find out, that there is an implicit name convention. Here is the yolo export as recommended:

yolo export model=yolov8n.pt format=edgetpu

Despite the format=edgetpu, this export function creates file names like this for instance:

yolov8n_integer_quant.tflite

But this will never be directed to the TPU. To run it on the TPU, it has to have a name with "_edgetpu.tflite" in the name, like

yolov8n_integer_quant_edgetpu.tflite

The missing link is the edge tpu compiler. But when you try to install this compiler on the Raspberry Pi as described here: Edge TPU Compiler, the installation fails. There are examples, where the compiler was made available for the Pi: Edgetpu compiler for ARM64. However, trying it out I got an error message "Internal compiler error. Aborting!", something was missing.

The path, that worked well in the end was compiling in Colab instead. There is a github repository with a Jupyter notebook: Compile for Edge TPU.

Clicking on the "Open in Colab" button will get will you to Colab.

To compile a file, just follow the description. Click on the file icon on the left.

The hit the upload icon, navigate to the location on your local machine and select the .tflite file to upload.

Then enter the file name into the code cell and hit "Run All" from the "Runtime" menu item.

Check the compiler output for results to see that there are operations assigned to the TPU. The compiler output file will be downloaded automatically to the Download folder of the local machine.

Here is my summary of all test runs. The CPU only results are coming from the "*.tflite" models and the TPU results from the "*_edgetpu.tflite" models:

Interestingly, the Jetson Nano wins with larger image sizes. But the RPi-Coral combinations catches up and is running away at smaller sizes. This will make this combination more suitable for smaller image robotics applications.

Originally I also assumed that the int8 model would be the fastest and most compact version to work with. But that isn't true at all from my observations. The compiler output stated:

Number of operations that will run on Edge TPU: 0
Number of operations that will run on CPU: 254

There might be some better solution for int8, but this was my result. I also noticed a drop in accuracy with int8, which is related to the low resolution of int8.<br>

In the end, the best results in accuracy are coming from the fully quantized integer version. It makes sense to only concentrate on that one.

To verify the results and install yolov8 plus USB Coral TPU, download an run the script:

wget https://github.com/StefansAI/Yolov8_Rpi5_CoralUSB/raw/main/scripts/yolov8_coral_install.sh
chmod +x yolov8_coral_install.sh
./yolov8_coral_install.sh

See my scripts here:Yolov8_Rpi5_CoralUSB

This script performs the installation, performs the exports of the yolo models and copying to "_edgetpu.tflite" versions. At the end of the script, the GardenCam videos and the GardenCam models are downloaded to demo the installation for few seconds each. The average python code then calculates the average exeution times for each run and displays the results.