Flying Drones in the Wind Really Blows

There is a lot of buzz these days around autonomous aerial vehicles (AAV) and all of the ways that they can benefit us in our everyday lives. From express deliveries to disaster management, search and rescue operations, and mapping of inaccessible locations, the list of potential applications goes on and on. But when was the last time a drone dropped off an online order at your home? If you are like most people, the answer is “never." While the potential of UAVs to transform our lives in many ways is real, the reason that relatively few of us have experienced that stems from a number of problems that have yet to be solved.

One of these problems is the difficulty of executing safe and precise flight maneuvers under windy conditions. At present, AAVs are typically either flown under near-ideal conditions, or a human operator has to step in to take the flight controls. It is not practical to require human operators of large drone fleets every time the wind gets a tad gusty, so a better solution is needed. Researchers are the California Institute of Technology have been working on this exact problem, and have just published their work, called Neural-Fly, that uses deep learning to help drones adapt to flying under any type of real-world weather conditions.

This is a very difficult challenge to take on. Wind patterns, which can be very complex and unpredictable, and how they are related to aircraft maneuverability is not well understood. Accordingly, traditional control design techniques that rely on classical physics-based methods tend to fall flat. It has been observed that varying wind conditions share a common representation, and the team leveraged that observation in building a domain adversarially invariant meta-learning model to learn that shared representation. The initial training of this model was completed with a paltry 12 minutes of flight data. Using a separation strategy, it was possible to adjust the pretrained representations by updating just a few key parameters. This property allowed the model to adapt to varying, real world wind conditions as they were experienced.

Validation of Neural-Fly was carried out at the CAST Real Weather Wind Tunnel where a ten by ten foot array of more than 1,200 individually-controllable fans generated complex wind patterns at up to 27 miles per hour. A drone equipped with Neural-Fly was programmed to continually travel in a figure-eight pattern during the tests. It was found that the error rate in following this path was 2.5 times to 4 times less than the current, traditional state of the art control techniques that do not make use of deep learning. Interestingly, the wind speeds encountered during validation were twice as fast as those encountered when collecting training data, indicating that the model is well generalized, and could likely deal with even stronger winds.

You might expect that Neural-Fly requires some serious computational resources that would price most people out of using the method. Surprisingly, that is not the case — Neural-Fly was implemented on the low-cost, low-power, and lightweight Raspberry Pi 4 single board computer. With the performance and adaptability of Neural-Fly, and the use of low-cost, off-the-shelf hardware, it offers real potential to expand the scope of real world drone applications in the near future.