Type on Any Surface with the TypeAnywhere Fingers-Only Keyboard
This QWERTY-based system uses finger-mounted sensors to simulate a keyboard with inputs registered by tapping fingers on any surface.
Computers, laptops and smartphones are just several devices we use that require a keyboard to provide text entry with certain apps. We’ve been using them for so long that they have become ubiquitous in our daily lives, but carrying around different keyboards for multiple devices isn’t practical or feasible with our on-the-go nature. But what if you could interface with every device simply by tapping your arm or leg? That’s the idea behind the University of Washington’s TypeAnywhere text entry tool.
TypeAnywhere is a QUERTY-based entry system that employs wearable sensors on two hands that detect finger-tap actions on any surface, a decoder that converts tap sequences into text, and an interface for text editing. The system is designed around the commercial Tap Strap1 wearable keyboard, which uses accelerometers for tap detection. “We feed a detected finger-tap sequence to a neural decoder modified from the BERT model, which then displays the output text on the typing interface,” state the engineers in a recently released paper. “Similar to Type, then Correct, we also designed a text correction interaction for TypeAnywhere that avoids the need for cursor navigation.”
The TypeAnywhere system doesn’t rely on position information to decode the intended letters being typed by the user; instead it relies only on the index of the finger being tapped and the context provided by the text entered. This allows the user to perform the tap at any location, so long as they use the correct finger.
Unlike other QWERTY-based text entry devices, the engineers were able to train the model without collecting data from extensive user studies. Instead, they garnered the training data by converting each letter in a phrase set to a corresponding finger ID using a collection of over 3.6 million samples and adapting the different finger-to-key mappings by altering the finger IDs in the training set. Testing showed that participants could achieve about 70 WPM (words per minute) tapping on a table and around 44 WPM while tapping on their laps.