RuneRing Performs Sophisticated Gesture Detection with Affordable Hardware

We’ve been promised gesture control for a long time, but when was the last time you used gesture control outside of a Nintendo game? Have you ever? While the basic sensor technology required for gesture control has been around for a long time, the implementation never seems to be very good. Devices are expensive and the actual gesture recognition tends to be unreliable. But it looks like Ambrush solved both problems with RuneRing, which performs reliable and sophisticated gesture detection with affordable hardware.

RuneRing is a gesture input device meant for use with regular old computers, tablets, and smartphones. It consists of a ring that a user can wear on the finger of their choice and a tethered bracelet worn on the wrist. The user can perform a gesture (waving their hand in the air like a wizard casting a spell) and the ring will classify that and perform an associated action. That action can be almost anything, but for maximum compatibility with different operating and software, it works best when sending a simulated key press, key press combo, or mouse movement.

For example, drawing an “F” shape in the air can type an “F” character on the connected device. But it doesn’t need to be that simple or reductive. Drawing an “F” could, for instance, send a hotkey combo that activates a macro, instead. If a keyboard or mouse can do it, RuneRing probably can, too — the user is just performing a gesture instead of tapping a key.

Ambrush was able to achieve this with surprisingly inexpensive components. Those include a Seeed Studio XIAO nRF52840 development board, a BMI160 IMU module, and a tiny 80mAh lithium battery from a set of wireless earbuds. The IMU goes in the 3D-printed ring, while the other components go in the 3D-printed bracelet.

But the real magic happens thanks to Ambrush’s machine learning model, which they built in TensorFlow and converted to TFLite to run on the microcontroller. The nRF52840 has relatively little processing power (when compared to a computer or server) for a machine learning model, but it is enough for this task. It looks at the incoming IMU data and compares that to the synthetic gesture data used for training. Through the dark sorcery of mathematics and machine learning, it can accurately associate a real-world gesture with a training data gesture. If the match is strong enough, it uses the nRF52840’s onboard Bluetooth adapter to send the action to the connected device.