Human Guidance in Interactive Program Aids Motion Planning for Robots

Carrying out tasks and motion planning in environments where not everything is clearly visible all the time has been a long-standing problem in robotics. A new strategy developed by engineers at Rice University — called Bayesian Learning In the Dark, or BLIND — seeks to resolve this. Combining Bayesian inverse reinforcement learning with established motion planning techniques, the novel algorithm keeps a human in the loop to assist robots with high degrees of freedom—that is, a lot of moving parts.

Bayesian inverse reinforcement learning refers to a technique by which a system learns from continually updated information and experience. Rather than programming a trajectory before a task is begun, BLIND inserts a human mid-process to refine the suggestions made by the robot’s algorithm. Information in the human’s head becomes part of the available information used to compute trajectories within a high-degree-of-freedom space.

To test BLIND, the Rice lab directed an articulated arm with seven joints — a Fetch robot — to grab a small cylinder from a table and move it to another, moving past a barrier to do so. When directing a human to perform such a task, simple instructions could be used. For instance, “lift your hand.” When directing a robot, however, programmers have to be specific about the movement of each joint at each point in its trajectory. This is especially true when obstacles block the machine’s perception of its target.

The system uses a specific way of feedback called critique. This is essentially a binary form of feedback in which the human participant is given labels on pieces of the trajectory. These labels take the form of connected green dots representing possible paths. As BLIND steps from dot to dot, the human approves or rejects even movement as a way of refining the path and avoiding obstacles as efficiently as possible. The process can be seen in a video posted by the lab.

One of the most important parts of the system is that human preferences are hard to describe via mathematical formulas. BLIND simplifies robot-human relationships by incorporating these preferences, which the researchers believe is how applications will get the most benefit from this work. Small amounts of targeted human intervention can significantly enhance the capabilities of robots in complex environments.

The paper detailing the methods and findings was recently presented at the 2022 IEEE International Conference on Robotics and Automation. Anticipated future work building on the study includes performing the evaluation with human users as well as exploring other methods of interjecting human guidance for high-degree-of-freedom robot learning.