TapID Transforms Everything Into Touch Interactive Devices

VR systems typically rely on cameras to track hands and controllers to trigger inputs in applications and games. VR tech has been branching into new and different areas over the last decade, ranging from pilot training in flight simulators to visualizing the latest architectural designs. The possibilities of utilizing that technology are virtually endless; however there has been little advancement into harnessing VR for everyday applications. Now, engineers from ETH Zurich are looking to break through that stagnation with TapID — a wrist-worn device that detects finger tapping on surfaces for touch interaction in VR environments.

The wrist-based inertial sensing system works in tandem with headset-tracked hand poses to trigger input in VR. The researchers designed TapID using a pair of inertial sensors embedded in a flexible strap, with one positioned at either side of the wrist. Data transmitted by those sensors allow TapID to detect surface-touch events and even the finger(s) used to do the tapping. The team found that their sensor design can detect minute differences in the vibration patterns on the wrist to differentiate between each characteristic finger movement.

A custom machine learning pipeline processes the collected sensor data in real-time. That information, combined with a camera system built into a set of VR glasses, captures the hands' position, which TapID generates into precision inputs. The engineers have demonstrated its capabilities in several applications, including a virtual keyboard and a piano, real-time photo manipulation, and interacting with the internet. Because TapID is worn as a wristband, it also has potential in portable applications for productivity on the go.