Surface Level Intelligence

Robots are playing an increasingly large role in our daily lives as the years go by. Today, you can routinely find robotic systems driving vehicles, vacuuming our homes, and assembling the products we regularly use in manufacturing facilities. But we are still a very long way from the dream of general-purpose robots that can do virtually anything that we ask of them.

It initially comes as a surprise to the majority of people when they find that the nuts and bolts of robots — such as the sensors, actuators, and mobile computing systems — are not the primary factor holding back progress. Rather, it is a software problem. In particular, the control algorithms that help robots to understand and interact with the world around them are all lacking in one way or another.

Thankfully we are well beyond the days in which robots were controlled by precisely programmed sets of rules. These systems are unable to adapt to new or unexpected scenarios, and the complexity of the algorithms left them able to only perform the most basic of tasks in highly structured environments. We now heavily utilize deep learning algorithms, which effectively allow robots to program themselves as they learn to carry out their tasks by example.

But when it comes to learning by example, challenges arise if the robot must replicate tasks on surfaces that differ geometrically from what was seen in the demonstration, or if demonstrations vary in timing or speed. To address these cases, a new framework was proposed by researchers at TU Wien and the Austrian Institute of Technology. Their technique, called Probabilistic Surface Interaction Primitive (ProSIP), incorporates surface geometry into learning, aligns tasks independent of time by using the surface path and local features, and projects tool motion onto the surface, making the control algorithm adaptable to different scenarios and robotic platforms.

The ProSIP approach offers a solution for robots learning complex interaction tasks, especially on freeform 3D surfaces, such as those found in polishing, sanding, and cleaning. Using ProSIP, tasks are modeled through the trajectory of a tool's center point along a specified path, while maintaining precise control of the interaction contact point and accounting for the geometry of the surface. This modeling is essential, as different surface shapes, like flat areas and sharp edges, require distinct approaches. The framework captures these variations by systematically integrating geometric surface data into the learning process, making it possible to replicate human demonstrations with high fidelity across varied surface geometries.

In one demonstration, ProSIP was applied to a robotic edge-cleaning task on bathroom sinks. An instrumented sponge, equipped with markers for optical tracking, enabled precise recording of human demonstrations on the sink's edge. The sink surfaces were reconstructed in high resolution, allowing ProSIP to capture the tool's trajectory along the sink’s curved edge while factoring in the surface’s local geometry. ProSIP generated a detailed model of the edge-cleaning motions, which was then adapted to new sinks with different geometries, including distorted shapes. This adaptation was achieved by projecting the learned tool motions onto the new surface paths, allowing the robot to clean edges accurately on unseen sink designs.

The experimental results, validated through simulations and real-world testing, demonstrated ProSIP's robustness and effectiveness in replicating human cleaning motions across various sink geometries. This may be just one small step forward, but in conjunction with other advancements, ProSIP may one day prove to be important in the development of a general-purpose robot.