This Adafruit BrainCraft Device Uses Machine Learning to Audibly Announce What It Sees

Most new technologies tend to be immediately patented, which makes them difficult for hobbyists to work with. Fortunately, machine learning has followed a somewhat unusual path. While there are certainly proprietary machine learning models out there, many of the most powerful and capable models are open source. That doesn’t, however, mean that they’re necessarily easy to use. Adafruit has been working valiantly to make the technology more approachable and recently released the BrainCraft HAT to accomplish that. To demonstrate how to take advantage of the Adafruit BrainCraft HAT, they have just published a guide on how to build a device that uses machine learning to audibly announce a description of whatever it sees.

The Adafruit BrainCraft HAT, as our own Gareth Halfacree covered a couple of months ago, is a Raspberry Pi add-on board that contains everything you need to start working with Google’s TensorFlow Lite. TensorFlow is a machine learning platform with an open source library of machine learning models that can be trained for a variety of applications. TensorFlow Lite is an optimized version designed to be run “on the edge” — meaning on the actual end hardware, as opposed to in the cloud or on a connected computer. The Adafruit BrainCraft HAT can be attached to a Raspberry Pi computer and contains virtually all of the hardware you need to utilize TensorFlow Lite. That includes a TFT display, a camera connection, a five-way joystick and buttons, stereo microphones, LEDs, and more. This guide explains how to utilize all of that glorious hardware.

In addition to the Adafruit BrainCraft HAT and a Raspberry Pi, you’ll need a Raspberry Pi Camera (either of the current versions), M2.5 hardware, a small tripod stand, a tripod ball head, and access to a 3D printer. You’ll only need to 3D-print a few small parts, which can be handled by any 3D printer. Assembly is as straightforward as it gets. After everything is put together, you can head over to this other guide on Adafruit to learn how to get the software setup. As programmed, this software will use a pre-trained TensorFlow Lite model to perform object recognition on whatever you point the camera at. It will overlay the screen with a short text description of whatever it detects, and you can connect headphones or a speaker for it to automatically handle text-to-speech and read that description out loud. If you’ve been curious about machine learning but haven’t known where to start, this project is perfect for you.