Drones Are Learning to Fly by Sight

Quadcopter drones may be among the most maneuverable aerial vehicles around, but when it comes to squeezing through tight spaces, they have a tough time. Flying through a narrow opening requires precise positioning and careful speed control, leaving little room for error. Even small disturbances can send a drone into a wall, branch, or other obstacle.

A group led by researchers at Zhejiang University has developed a new approach that enables drones to fly through narrow openings without loading them down with lots of bulky sensors. Instead of relying on a traditional robotics stack built around state estimation, gap detection, trajectory planning, and control, the team trained a neural network to directly convert camera images and other onboard sensor data into low-level flight commands. This system allows a quadcopter to make split-second decisions and perform aggressive maneuvers that would normally require carefully engineered software pipelines.

The team’s work takes inspiration from birds. Birds routinely fly through dense forests, narrow branches, and other confined spaces without explicitly calculating trajectories or building detailed maps of their surroundings. The researchers wanted to determine whether a drone could learn a similar sensorimotor behavior, reacting directly to what it sees rather than relying on a chain of separate algorithms.

To test the idea, the team built a custom quadrotor measuring 38 cm wide and 10 cm tall. The aircraft carried a monocular camera, a PX4 flight controller, and an NVIDIA Jetson Orin NX computer. Camera images were processed by a reinforcement learning policy trained in simulation before being transferred to the real world.

The biggest challenge was teaching the system how to discover successful flight strategies in the first place. Flying through a narrow gap leaves very little margin for error, making random trial-and-error learning inefficient. To overcome this, the researchers used trajectories generated by a model-based planner to guide the learning process and help the neural network explore feasible solutions.

During testing, the drone successfully flew through a rectangular opening measuring 60 cm by 20 cm, leaving only 5 cm of clearance on either side of the vehicle. The policy automatically determined the best orientation for passing through the gap, often rolling the aircraft sharply so that its narrow dimension aligned with the opening.

The system handled gaps tilted at various angles, including some rotated as much as 90 degrees. In many trials, the drone rolled nearly onto its side while maintaining enough control authority to safely pass through the opening. The researchers reported near-perfect success rates for many of the tested configurations.

Even more interesting was the system's ability to react to moving targets. Although the neural network was never specifically trained on dynamic gaps, it successfully adjusted its flight path when researchers rotated or translated a handheld opening during flight. The drone tracked the moving target and altered its trajectory in real time to complete the maneuver.

The team also demonstrated flights through multiple closely spaced gaps and openings with different shapes, including triangular and parallelogram designs. Because the policy learned directly from experience, it did not require handcrafted visual features or manually defined traversal poses for each geometry.

The researchers believe the technology could eventually help drones operate in environments that are currently difficult to access, such as collapsed buildings, tunnels, caves, industrial facilities, and other cluttered spaces where precise navigation is essential. As these learned control policies continue to improve, future drones may be able to exploit their full maneuvering capability in ways that more closely resemble the agility of birds.