Skywriting

Turn thin air into a touchscreen for your smartwatch, with no additional hardware required, by using AirText.

Nick Bild
25 days agoMachine Learning & AI
Writing in mid-air (📷: Y. Gao et al.)

Smartwatches have a lot of utility in everyday life, handling tasks such as texting, answering phone calls, and tracking fitness. But when it comes to entering data into a smartwatch, there is much to be desired. The downside of having the device always in front of you by attaching it to your wrist is that only one hand is free for interacting with it. And anyway, those screens are tiny — typing out more than a few words is very slow and error-prone. This state of affairs leaves many smartwatch users begging for a better way to interact with their devices.

A new idea developed by researchers at China’s Zhejiang University promises to make smartwatches easier to use by providing a natural interface that does not require any additional hardware. AirText is an app that runs on a wide range of smartwatches, and allows users to “type” by tracing letters in the air with their finger.

AirText works by capturing readings from the inertial measurement unit (IMU) already present in nearly all smartwatches. IMUs record measurements of movement and acceleration as the wrist moves. Since moving one’s arm about while wearing a watch prevents viewing the display screen, the team designed the app such that the user does their air writing without rotating their wrist, which keeps the display in sight.

Since the IMU is mounted on the wrist, and not fingers, it is not able to capture all movements that occur while tracing out letters; the wrist and fingers can move in different, independent motions during this process. So, the first step was to correlate wrist motions with the motions associated with writing each letter.

This was accomplished by using a program called Leap Motion, which tracks finger motions using infrared sensors. Data from the IMU was paired with the infrared data as eight volunteers spelled out over 25,000 characters on five different smartwatches. Using the infrared data as the ground truth, a machine learning classifier was trained to translate IMU measurements into air-drawn letters.

During evaluation of the method, volunteers were able to type at over eight words per minute, with a word error rate ranging from 3.6% to 11.2%. The speed may be improved by making use of a word prediction program, in which users can tilt their smartwatch left or right to select a suggested word. To help fix the inevitable errors, a user can backspace by simply shaking their wrist.

AirText is currently limited in that it requires a short pause between entering each individual character. The team is currently exploring ways to get around this drawback to allow faster data entry. With this refinement, and perhaps also some work to reduce the error rate, the researchers may be able to achieve their goal of commercializing AirText.

Nick Bild
R&D, creativity, and building the next big thing you never knew you wanted are my specialties.
Latest articles
Sponsored articles
Related articles
Latest articles
Read more
Related articles