Eloquent Arduino's Benchmarking Puts the Raspberry Pi Pico Bottom of the Pile for TinyML Performance
Its low cost and feature set have made the Raspberry Pi Pico a popular device, but benchmarking shows it's not great for TinyML.
Eloquent Arduino's Simone Salerno has put together another head-to-head analysis pitting popular microcontroller boards against each other for TinyML performance — and this time the Raspberry Pi Pico is included in the line-up.
Running machine learning workloads on low-power, low-cost microcontrollers - known as "TinyML" - is becoming increasingly popular, and the low $4 cost of the RP2040-based Raspberry Pi Pico makes it a tempting target for experimentation. Iwatake's work using TensorFlow Lite to give a Raspberry Pi Pico number recognition capabilities is just one example — but Salerno's analysis of the RP2040's performance at these tasks does raise questions about the Raspberry Pi Pico's suitability for such workloads.
In his latest benchmark analysis, which follows one published three months ago comparing just three devices, Salerno pitted seven boards against each other for TensorFlow Lite workloads: The Arduino Nano 33 BLE Sense with its Arm Cortex-M4 processor running at 64MHz; the Espressif ESP32 with its 240MHz dual-core Xtensa processor; the Adafruit Feather M4 Express and its Arm Cortex-M4F at 200MHz; the STMicro STM32 Nucleo H743ZI2 with a Cortex-M7 running at 480MHz; the Teensy 4.0, the fastest board on paper with a 600MHz Cortex-M7; and the Raspberry Pi Pico, whose RP2040 includes two Arm Cortex-M0+ cores running at 125MHz stock.
"How does Raspberry Pi Pico perform on TinyML," Salerno asks: "No good (compared to its big brothers.)" Based on three fully-connected networks — a single-layer with 10 neurons, a two-layer with one 10-neuron layer and one 50-neuron layer, and a 10-layer with 10 neurons each - the Raspberry Pi Pico was far and away the slowest device on test - even below the Arduino Nano 33 BLE Sense, which runs at around half the clock speed but with a more powerful processor architecture.
Even accounting for the Raspberry Pi Pico's low cost, the results disappoint: The Espressif ESP32 board comes out on top for price-versus-performance, with Salerno indicating that his particular example was purchased for under $4 — less than the recommended retail price of the Raspberry Pi Pico. For those looking for the best performance, meanwhile, the Teensy 4.0 unsurprisingly tops the table.
The full write-up, including raw benchmark results, is available on the Eloquent Arduino website.