Fabriccio Doppler-Based Touchless Sensor Brings Flexible Gesture Sensing to Interactive Fabrics

Researchers at Dartmouth College, Beihang University, Shanghai Jiao Tong University, Wuhan University, and Microsoft Research have published a paper detailing a touch-free, gesture sensing technique for fabric-based wearables: Fabriccio.

"We present Fabriccio, a touchless gesture sensing technique developed for interactive fabrics using Doppler motion sensing," reads the abstract to the team's paper. "Our prototype was developed using a pair of loop antennas (one for transmitting and the other for receiving), made of conductive thread that was sewn onto a fabric substrate. The antenna type, configuration, transmission lines, and operating frequency were carefully chosen to balance the complexity of the fabrication process and the sensitivity of our system for touchless hand gestures, performed at a 10cm distance."

The loop antennas — chosen from a shortlist of four possible designs — measure just 9mm in diameter and are spaced 15mm apart. These antennas, plus the electrical connectivity to complete the circuit, are manufactured using Liberator 40 silver-coated Vectran thread — again picked from a shortlist of four possibles ranging from thin stainless steel to smooth conductive thread — stitched using an unmodified embroidery machine.

The electronics are made up of a customised sensing board split into two parts: A modified HB100 Doppler Radar Motion Detector forms the sensor half, while modified Adafruit Bluefruit LE Micro development board handles data collection and transmission to a remote laptop for data processing via the Random Forest machine learning algorithm.

The team built and tested a series of prototypes based on the technology, using a 10-participant cohort to attempt a total of 11 touchless and one touch-based gestures on fabric systems including interactive furniture, interactive clothing, and "soft things" like a backpack with integrated controls for a smartphone and interactive children's toys.

"We demonstrate that our system can achieve a 92.8 percent cross-validation accuracy and 85.2 percent cross-session accuracy in a user study with 10 participants and 11 touchless gestures as well as 1 touch gesture," the team concludes. "For the subset of seven gestures, the cross-user accuracy can reach 87.6 percent.

"Our technique provides a useful addition to existing sensing techniques for user input on soft fabrics, primarily based on touch and deformation. This enables a new set of applications on everyday objects that are covered or made of interactive fabrics. We believe our technique may serve as important groundwork for integrating the gestural input into the soft objects in people’s daily life."

The paper, presented as part of the ACM CHI Conference on Human Factors in Computing Systems (CHI'20), is available under open access terms via Microsoft Research.