How Does This Grab You?

There are many challenges that we face in the present age, the greatest of which is arguably finding where the television remote control has disappeared to. We all know the feeling — the couch cushions have been searched three times, nothing but dust bunnies turned up under the furniture, and anything the remote could be hidden under has been upended and tossed about, making the living room look like the scene of an FBI raid. Then just when you are fully willing to accept the reality of some freak physical phenomenon that caused your remote to be spontaneously transported to an alternate dimension, it appears right under your nose.

Some bright minds at MIT have turned their attention to the problem of quickly finding missing objects, such as lost remote controls or car keys. Their method, called RFusion, blends information from a camera, a radio frequency (RF) receiver, and a machine learning algorithm to handle the task.

RFusion is a robotic arm, with a gripper for picking up items. Objects that it locates must have an RFID tag attached to them for the device to detect. This information, fused with visual imagery from the camera, helps the robot to reduce uncertainty about the location of the object of interest, but cannot provide a precise location. The likely candidate locations are further inspected by the robot arm with the guidance of a reinforcement learning network that determines the optimal moves for the robot to make to get a better view of the object, and ultimately to pick it up and validate that the correct item has been retrieved.

Because RFusion uses RF signals, which can travel through objects, to locate items, line of sight is not needed. The device can find targets that are completely hidden from view. And it does so quite well — in a series of trials, RFusion was found to successfully locate objects that were fully occluded from view in 96% of cases.

The device is currently in the prototype stage, and not quite practical for household tasks. While the arm can pivot around to grab an object within reach, there is no provision for the robot to move about such that it can search a larger area beyond its immediate reach. Further, it is only capable of locating objects that have RFID tags attached to them, which is a bit of an inconvenience in domestic settings.

These current limitations make RFusion a bit more practical for industrial applications at this time, but the team is interested in incorporating RFusion into future smart home designs to help with any number of tasks, so be on the lookout for future enhancements.