Robotic Hide and Seek Champion
LAX-RAY teaches robots how to efficiently search for objects that are hidden from view.
Robotics has many applications in, for example, warehouses, retail stores, and pharmacies, where it can automate mundane tasks like picking items from shelves. When those shelves are neatly ordered, with each item in a precise, predefined location, it is relatively easy to program a robot to pick items on demand. But in cases where items are only known to be in a general location, the problem gets much harder. And if the needed item is obscured or completely hidden by other nearby objects, then the problem gets harder still.
These problems were taken on by a collaboration between researchers at UC Berkeley and Google. Their method, Lateral Access maXimal Reduction of occupancY support Area (LAX-RAY), teaches robots efficient search algorithms to quickly reveal occluded objects stored on shelves.
LAX-RAY starts by capturing an RGB image and a depth image, from which a distribution of where the search target is hidden is calculated, with the help of a pre-trained neural network. Incorporating information about previous states of the items, a motion plan for the next action is determined, then executed by the robot.
Three types of search policies have been developed — Distribution Area Reduction (DAR), Distribution Entropy Reduction over ‘n’ steps (DER-n), and Uniform. Each takes a somewhat different approach to compute a set of steps required to reveal a hidden target object.
The team developed a Python-based simulator called First Order Shelf Simulator (FOSS) to assist in testing the new method. Eight hundred random environments of varying complexity were generated with FOSS. In the simulated environment, each of the three search policies achieved a success rate greater than 97% in under two steps, on average.
Validation was also performed using a physical robot. In this case, using the Uniform and DAR searches, 60% and 80% success rates were achieved, respectively, within four steps. DER-n searches achieved a 100% success rate with approximately three steps on average.
At present, LAX-RAY can only move objects on a plane perpendicular to the axis of the camera, which limits the types of movement plans it can carry out. The team is working on adding more sophisticated depth models, as well as pneumatically-actuated suction cups, to allow for movements parallel to the camera axis.