tinyML Reaches New Heights with Crazyflie Nano Drone AI-deck

Machine learning in the skies is nothing new — for example, many entries in Hackster's first HoverGames contest incorporated ML, but what happens when your drone is only 27 grams (about an ounce) and runs on a 250mAh battery? The platform in question is the Crazyflie 2.1 from Bitcraze AB, and the answer comes in the form of the AI-deck, now available in early access. Decks are akin to shields for Arduino, or HATs for Raspberry Pi — expansion boards based on a specific header convention. So, by attaching an AI-deck to a Crazyflie 2.X, you end up with a 30-gram, multi-core, vision-enabled, Wi-Fi-equipped smart nano drone!

In order to provide sufficient computational power for running Convolutional Neural Networks (CNNs), without draining too much battery power from the tiny onboard LiPo battery, the Bitcraze team selected the GAP8 IoT application processor from GreenWaves Technologies, delivering 10 of an available 22.65 Giga (billion) Operations Per Second (GOPS) while consuming less than 0.1% of the total system energy. The AI-optimized GAP8 features a fabric controller (FC) core for control, communications and security functions, a compute cluster of eight cores, and ninth cluster core in the form of a Hardware Convolution Engine CNN accelerator (HWCE). An additional 64Mb of RAM and 512Mb of Flash on the AI-deck ensures sufficient space for running complex models, and an ESP32-based NINA-W102 gives the ability to stream images from the drone's ULP camera to a computer for training CNNs.

A look at its key features:

• GAP8 Ultra-low power 8+1 core RISC-V IoT application processor
• Himax HM01B0 Ultra low power 320×320 Bayer RGB camera
• 512 Mbit HyperFlash
• 64 Mbit HyperRAM
• ESP32-based NINA-W102 802.11b/g/n Wi-Fi
• Two Cortex-M 10-pin JTAG for GAP8 and ESP32
• ESP32-connected user button for e.g. UART bootloader
• GAP8-connected green LED
• ESP32-connected green LED
• 4.4g
• 30x52x8mm (WxHxD)
• Standalone mode for use without Crazyflie 2.X

Schematics, examples, and documentation can be found in the project's GitHub repo, and as an early access product, users will need to be familiar with JTAG programming/debugging until the software reaches a more stable state. The AI-deck and Crazyflie 2.1 are now up on the Bitcraze store for $195 each.