Crash Course in Avoiding Crashes

Assistive aerial teleportation (AAT) is a concept that technology has just begun to catch up with in recent years. The idea is that an operator can pilot a drone as if they were actually physically flying in the vehicle. But as you might expect in a new and rapidly growing field, there are still some issues to be worked out.

One of these issues is that AAT control systems do not factor in what the operator is looking at, which can reveal their intentions. Further, these systems typically do not have the ability to perceive their environment, and use that information to inform flight plans. Operator intentions can be leveraged by the control system to smoothly achieve the operator’s objectives, and environmental awareness can help to refine the flight path into a safe, feasible trajectory. A team at Zhejiang University has taken on the challenge of solving these present shortcomings of AAT control systems through a system that they have integrated into a quadcopter drone.

The system, called gaze enhanced perception-aware safe assistive aerial teleoperation (GPA-Teleoperation), takes both remote control input, as well as operator eye movements as controls for the vehicle. These are combined to produce a topological path that matches the operator’s intention. An optimizer then incorporates environmental information to generate a safe, collision-free trajectory.

A headset shows operators a video feed from a camera onboard the drone. Eye trackers within the headset are used to guide the direction of the vehicle. The remote control unit only has a single channel to control speed, much like the accelerator of a car. These inputs feed into the vehicle’s control system, where a custom algorithm determines the optimal trajectory for the drone to take to achieve the operator’s objectives.

To evaluate GPA-Teleoperation, the team recruited a number of participants, primarily with little flight experience. They were briefed on the task to complete and given a short practice session. In one trial, GPA-Teleoperation was compared with traditional remote control-only operation. It was found that the new method significantly enhanced vehicle stability, likelihood of success, and also decreased time required to complete the task.

In another trial, the drone was flown both with, and without, using the perception-aware feature to determine how it impacts flight safety. Volunteers were asked to fly the quadcopter through a maze that contained surprise objects hidden throughout. It was found that the perception-aware feature turned corners on a safer trajectory that allowed it to see the hidden objects sooner. In general, turning on the feature kept more distance between the objects and the drone than was observed when it was turned off.

GPA-Teleoperation makes drones easier to fly for unskilled users, and increases flight safety. With future enhancements to extend the technology to dynamic environments, it may find itself in a wide variety of applications.