Virtual Worlds and Real Objects with MemGlove

Hand tracking and stimuli sensing are two of the perennial challenges of Augmented Reality systems — though there’s no shortage of work on that challenge, there’s just so many complex scenarios and situations that make a one-size-fits-all solution tough to find.

However, researchers from MIT CSAIL have a very versatile solution of their own. It might not be a one-size-fits-all solution, but certainly is one-size-fits-a-whole-lot. The “Mens et Manus Glove,” (a pun on MIT’s motto, which means “Mind and Hand”) is a pose-detecting smart glove that can also discern between over 30 common objects. The team says the glove is built for future applications like “a video game where you can grab an object of your desk and have it be seamlessly incorporated into gameplay.”

Though this is not the first glove to aspire toward similar applications, the real stand-out part of the system is in the details of the design and fabrication. And in that space, Mens et Manus Glove (MemGlove) is unlike its counterparts in two big ways — first that it costs just about $60 to fabricate, and second that it contains no embedded electronics. This means sensing and data transfer come primarily from soft textile based communications through knitted and sewn fibers.

Beyond the positional data the glove gathers, it also captures biometric data bout the user (heart rate), and data about the objects in-hand (temperature, pressure, and conductibility). Between all this information the glove perceives, a machine learning model is able to discern fine-motor gestures with high granularity — it's even able to identify which letter of the alphabet is being written at any moment. The key to this is their explorations of combining resistive sensing and fluidic pressure sensing as opposed to conventional physically hard electrical components. These two sensing types, when coupled, unlock significantly more sensing functionality than the sum of their performances separately.

In the paper associated with their work, the team points out a few worthwhile spaces for future work. Most notably, the glove currently works with single domain inference tasks, which is to say, hand pose, temperature, and holding force can each be discerned independently, but not together, as each of those outputs uses similar signal data. But that even with its limitations, the authors say MemGlove’s robustness, versatility, and accuracy make it a great contender in unlocking exciting augmented reality applications across many fields.

More on this project can be found in the paper A Simple, Inexpensive, Wearable Glove with Hybrid Resistive‐Pressure Sensors for Computational Sensing, Proprioception, and Task Identification, in Volume 2, Issue 6 of Advanced Intelligent Systems.