"Deep Drone Acrobatics" Teaches NVIDIA Jetson TX2-Based Drones Risky Tricks in Simulation Only
Trained purely in simulation, the real-world drones prove capable of a range of tricks — including the 3g Matty Flip.
A team of researchers at the University of Zurich and ETH Zurich have published a paper detailing "deep drone acrobatics" — an AI system trained purely in simulation running on an NVIDIA Jetson TX2 computer-on-module capable of guiding its host drone through the trickiest of aerial maneuverers: the Matty Flip.
"Performing acrobatic maneuverers with quadrotors is extremely challenging. Acrobatic flight requires high thrust and extreme angular accelerations that push the platform to its physical limits," the team explains of its work. "Professional drone pilots often measure their level of mastery by flying such maneuverers in competitions.
"We propose to learn a sensorimotor policy that enables an autonomous quadrotor to fly extreme acrobatic maneuvers with only onboard sensing and computation. We train the policy entirely in simulation by leveraging demonstrations from an optimal controller that has access to privileged information. We use appropriate abstractions of the visual input to enable transfer to a real quadrotor."
The simulation-based training takes place in the popular Gazebo simulator, with the RotorS extension for high-fidelity quadrotor physics modelling. Once training is complete, the resulting model is moved to a custom 1.15kg quadrotor with a 4:1 thrust ratio, an on-board Jetson TX2 computer-on-module, and an Intel RealSense T265 camera.
"We show that the resulting policy can be directly deployed in the physical world without any fine-tuning on real data," the team explains. "Our methodology has several favourable properties: It does not require a human expert to provide demonstrations,it cannot harm the physical system during training, and it can be used to learn maneuvers that are challenging even for the best human pilots. Our approach enables a physical quadrotor to fly manoeuvrers such as the Power Loop, the Barrel Roll, and the Matty Flip, during which it incurs accelerations of up to 3g."
"The Matty Flip is probably one of the manoeuvrers that our approach can do very well, but human pilots find very challenging," project member Antonio Loquercio explains in an interview with IEEE Spectrum. "It basically entails doing a high speed power loop by always looking backward. It is super challenging for humans, since they don’t see where they're going and have problems in estimating their speed. For our approach the manoeuvrer is no problem at all, since we can estimate forward velocities as well as backward velocities."
The team's work has been published as part of the Robotics: Science and Systems 2020 conference under open-access terms; Loquercio indicates that the next step is to extend the manoeuvrer duration beyond the currently-tested limit of 20 seconds.