Going Up?

Drones are now frequently used in applications that include smart agriculture, defense, construction site monitoring, and environmental monitoring. As these, and other applications of the technology continue to expand, the need for autonomous systems to pilot unmanned aerial vehicles (UAV) is rapidly increasing. However, due to the tight energy, weight, and space constraints of drones, building on-board, practical, production-ready autonomous systems remains a challenging proposition.

A trio of engineers at the University of Trento in Italy have taken advantage of recent advances in embedded systems to build an energy-efficient tinyML system for inferences on the edge. In particular, the team has fitted a DJI Tello Micro Aerial Vehicle with an inexpensive, low-power, and lightweight OpenMV Cam H7.

The OpenMV Cam H7 sports a 480 MHz STM32H743VI Arm Cortex M7 processor, with 1 MB of SRAM and 2 MB of flash memory. An on-board OV7725 image sensor can capture images for machine vision algorithms at up to 640 by 480 pixels of resolution. OpenMV Cam is also supported by TensorFlow Lite for Microcontrollers and Edge Impulse, allowing custom, pre-trained machine learning models to be loaded onto the device.

To test the concept, the engineers built a fully autonomous drone control system that was capable of carrying out a set of objectives. In this case, the control system captures images of a crowd to determine if people are wearing face masks, and repositions itself as needed to get a better view. A MobileNet V2 architecture convolutional neural network was trained on the open source Face Mask Lite dataset to classify images (i.e. wearing mask, not wearing mask). A custom algorithm was implemented to handle acquiring targets.

After quantization from a 32-bit floating point to an 8-bit integer, the model was 585 KB in size, fitting comfortably within the OpenMV Cam’s flash memory. Likewise, the 296 KB of RAM required was well within bounds for the device. While not exactly snappy, the 1.7 second inference time is reasonable for the task. So it’s small, but does it work, you ask? The validation showed that it works quite well, with an average classification accuracy of 97%.

Attaching a consumer-grade development board to a tiny drone may not be earth-shattering news, but the implications are very important, nevertheless. This work has shown how inexpensive, and relatively simple it can be to build and program an autonomous aerial vehicle that is capable of performing useful work in real world scenarios.