AI Data Glove Turns Your Hand Into a Keyboard!
Zack Freedman's cyberpunk device was trained with TensorFlow as a usable head-mounted display input.
Earlier this year, we reported on the Somatic Data Glove by Zack Freedman, which aimed to “eventually be able to operate any wearable computer with a heads-up display.” That time is now, and as shown in the video below, his glove's hardware is finally capable of interfacing with any computing device, wearable or not.
The AI data glove utilizes a hand/finger-mounted sensor unit, comprised of a 9DOF IMU next to the index finger, plus four magnets that move along with one’s fingers. These activate and deactivate Hall effect sensors on the knuckle, providing on/off inputs without the unwanted restriction of a force-sensitive resistor or similar.
These sensors are fed into the cyber-wristband, which features a Teensy 4.0 for processing (underclocked to 150 MHz), along with a Bluetooth module that provides the HID link to the controlled device. A 340mAh battery is also onboard to make it truly wireless.
Importantly, the glove doesn’t try to reproduce one’s hand in 3D space, but instead uses the magnetic sensors to put it in different input modes. The IMU then produces specific actions, like mouse movement or finger handwriting text input via a TensorFlow Lite model. The device is able to run its 600 node model — trained with over 10,000 handwriting samples — with a sub-10ms response time.
Assembly info is found in Freedman's project write-up, and electronics, software, and printable models are available on GitHub.