Ashish Patil's TinyML Raspberry Pi Pico Project Handles Handwritten Digit Recognition On-Device
Using CircuitPython, this compact machine learning project takes camera input and looks for digits to classify.
Software developer Ashish Patil has put together a step-by-step guide to computer vision work on the Raspberry Pi Pico, processing the image from a camera sensor to recognize handwritten digits.
"[This is] a project using Raspberry Pi Pico, an [Omnivision] OV7670 camera module, a 120×160 TFT LCD display, and machine learning" Patil explains of his creation, "to build a portable handwritten digit classification system. [It] analyzes photos received from a camera and tries to infer what digit was present in the image."
Traditionally, machine learning models are trained and run on high-end power-hungry hardware. In recent years there's been a demand for the ability to run these workloads at the edge, rather than in a data center, leading to the logical conclusion: TinyML, in which machine learning models can be run on even resource-constrained microcontrollers.
"It is important to note that our machine learning model can be executed entirely on a Raspberry Pi Pico; a connected computer or cloud are not required," explains Patil. "Therefore, creating compact machine learning models that fit in Pi Pico's RAM and is accurate enough for our work is a major challenge we are aiming to overcome. This is no easy task."
Patil's project is written in CircuitPython, and while only tested on the Raspberry Pi Pico should in theory work on any CircuitPython-compatible device with enough free general-purpose input/output (GPIO) pins for the camera module and TFT display.
"The placement of the camera and the subject has a significant impact on the output," Patil notes. "With an LCD, it is easy to align your handwritten numbers with the camera as you can constantly see what your camera is seeing."
The model itself is trained on the MNIST handwritten digit dataset on a desktop or laptop before being exported to a format suitable for use with the Raspberry Pi Pico. Once installed, the resulting code automatically process the camera image and attempts to find and classify numbers — though, Patil admits, "the code is highly experimental [and] some tinkering will still be necessary to get it to work."
More details are available in Patil's blog post; the source code has been published to GitHub under an unspecified open-source license.