EPFL Researchers Developed Deep Learning Algorithm to Create Tiny Fly-Like Robots

Researchers at EPFL (Ecole Polytechnique Federale de Lausanne) have developed a deep learning motion capture algorithm that utilizes multiple camera views to capture a fly’s movements in 3D, to use that data to build fly-like robots. The idea sounds crazy; until you think about how agile they are when walking on any surface, flat or otherwise. Now imagine applying that agility and movement for robots that could traverse any terrain.

EPFL professor Pavan Ramdya (Brain Mind Institute) explains, “Unlike most vertebrates, flies can climb nearly any terrain. They can stick to walls and ceilings because they have adhesive pads and claws on the tips of their legs. This allows them to basically go anywhere. That's interesting also because if you can rest on any surface, you can manage your energy expenditure by waiting for the right moment to act.”

It’s with that idea that Ramdya and his team developed DeepFly3D — a motion capture system designed to grab motion data from the fly Drosophila melanogaster, which they state is a model organism that’s a favorite among biologists. The system is comprised of seven cameras that record the fly as it sits atop a tiny floating ball that rotates in different directions like a treadmill. The fly is glued to an immovable prop that remains stationary while the ball turns, which is done to grab accurate information on the fly’s movements.

That information collected from the camera images is then processed by the DeepFly3D software, which can infer the 3D pose of the fly, meaning it can automatically predict and make behavioral measurements at high resolutions for any number of biological applications. Ramdya goes on to state, “The fly, as a model organism, balances tractability and complexity very well. If we learn how it does what it does, we can have important impact on robotics and medicine and, perhaps most importantly, we can gain these insights in a relatively short period of time.”