GhostAR Records Human Movements for Playback and Collaborative Robot Programming

Robots have been used extensively in manufacturing, normally to interact with inanimate object over and over in a predefined path. According to the research paper found here, as robotics continue to advance and infiltrate into broader commercial and consumer markets, this will likely mean humans and robots working collaboratively together.

In order to facilitate this type of interaction, researchers at Purdue University have been developing the GhostAR workflow. This system allows users to first demonstrate the human task, for instance putting an object in a designated spot as seen in the video below. These movements are recorded, then rendered as an AR ghost, enabling a human robo-minder to replay these actions as a sort of out-of-body experience, and set up the robot coworker to do its task using the ghost AR image as a visual reference.

For their research, the team is using an Oculus Rift Headset, plus IR LED sensors to track the motion of a human body. A pair of Oculus Touch controllers is used to interact with the system in AR. Three robots have been designed to help with this interaction, including: GripperBot, an omni-bot with a 6DOF robot arm on top; Armbot, a fixed 6 DOF robot arm (namely the Arduino Tinkerkit Braccio); and Cambot, an omni-wheel robot with a mounted camera. An NVIDIA Jetson TX1 Development Kit running ROS is implemented to control the mobile robot base along with an Arduino.