If at First You Don’t Succeed, Try AI Again

Building machine learning-powered robots that can perform a wide range of tasks in diverse environments poses significant challenges for engineers. One of the primary difficulties stems from the reliance on a limited set of training data. Machine learning algorithms require extensive and representative training data to learn and generalize effectively. However, as environments change and differ from the data they were trained on, robots will inevitably encounter difficulties in adapting their learned behaviors to new situations.

One challenge arises from the concept of domain shift, where the distribution of data in the training environment differs from the real-world environment where the robot is expected to operate. This discrepancy can lead to poor generalization, causing the robot to struggle with tasks it hasn't encountered during training. For example, if a robot is trained to recognize objects in a controlled laboratory setting and is then deployed in a cluttered and unstructured real-world environment, it may struggle to accurately identify and interact with objects due to the differences in lighting conditions, object variations, and occlusions.

This is arguably the largest factor that is preventing the widespread adoption of personal robots that assist us with mundane household tasks. As such, there is a tremendous amount of interest in moving the state of the art forward. Towards that end, a team led by researchers at MIT has developed a framework that can help robots to quickly learn to adapt their skills to new environments. And best of all, that adaptation process can be driven by non-technical users of the system.

Initially, a robot is trained using typical methods, in which a large set of training data is leveraged to generalize the algorithm as well as possible. After the robot is deployed to a new environment and encounters a situation that trips it up and causes a failure, it enters an interactive learning mode.

This process begins with the robot showing the user the specific task that caused it to fail in carrying out its actions. Next, the user is prompted to perform that task while the robot observes them. Then, the robot searches through all features in the space and generates counterfactual examples that show what would need to change for it to succeed.

At that time, the user is asked to provide feedback about what elements are not important in completing the task. For example, the color of a cup may make no difference when the task is to pick up a cup. This feedback is then leveraged to create a new dataset, using data augmentation techniques, to fine-tune the model. For example, in the case of the cup, the framework may alter the color of the cup in many ways such that the model learns to recognize features other than color as the important ones.

By being walked through this process, the user need not have any special skills in machine learning or robotics to improve the system. And through the use of data augmentation, a single demonstration of a task may be sufficient for generalization in a new environment.

The new framework was tested in a simulated environment to validate the team’s methods. A number of tasks were attempted, like picking up an object and placing it on a table, or locating a key and unlocking a door. Human participants were recruited to work within this environment to help the simulated robots adapt to new situations. It was discovered that using this approach, the robots were able to learn more quickly than was the case with other methods. Fewer demonstrations of the tasks were needed as well.

In the future, the researchers want to validate their system on physical robots in real-world environments. They also intend to work towards making the process of generating augmented data and fine-tuning the model faster. Such improvements might lead to the development of more practical general-purpose robots in the future, even in residential settings.