Swiping Under the Radar

All of the digital devices now offering interaction through gesture or voice can leave a person feeling like keyboards are so20th century. Who wants to press buttons when they can get what they want with a wave of the hand? Working with computers is now just like it was imagined on Star Trek, right?

Well, not exactly, but we are getting closer. Take gesture recognition, for example. This is commonly done with cameras, which makes for higher cost, high power consumption devices that may compromise the privacy of their users. This is not ideal, and for many use cases it is completely impractical.

This has led some to search for another path forward, as is the case with a research group at Google that is working towards a more efficient gesture recognition solution based on radar sensors. The solution, dubbed RadarNet, is capable of recognizing four directional swipes and an omni-swipe using a 6.5 × 5mm radar chip that has been integrated into a smartphone.

The radar chip is a new Soli chip design with one transmitter and three receivers. This is a decrease of one transmitter and receiver when compared with the previous Soli chip, which allowed the footprint to shrink by more than 75%. While this change did lead to a decrease in the signal to noise ratio, it made it possible to include the chip in a smartphone.

Interactions are based on the recognition of mid-air swipe gestures, which are designed to resemble natural movements that are performed on physical objects, like swiping things away. Each gesture was designed to be simple to perform with many potential use cases.

These gestures are recognized by first reflecting 25 Hz bursts of radar transmissions off of the device user. Signal processing algorithms convert these raw signals into complex range Doppler maps. Complex range Doppler maps contain phase information about the angular position of targets, which make them more suitable for detecting directional gestures than the more commonly used absolute range Doppler maps.

Gesture classification is accomplished via a custom convolutional neural network that has a model size and inference time about 1/5000th that of existing radar gesture recognition techniques. The reduction in computational and power requirements make this model suitable for resource-constrained mobile computing applications. The final output consists of three separate classifications: a portrait prediction (right, left, or background), a landscape prediction (up, down, or background), and a omni prediction (omni or background).

Gesture data was collected from a total of 7,647 participants to train the machine learning model. An additional 285 hours of data was collected in which participants were interacting with their phones without performing gestures. The trained model achieved an average gesture recognition accuracy of over 90%.

At present, RadarNet is only able to recognize five gestures, which limits the applications in which it can be used. The team believes that it will be possible to recognize more gestures with the appropriate training data, but that remains to be confirmed in the future.

The researchers were able to collect a huge body of training data to support their research, but in many cases this is not feasible. Take a look at our recent article that details how to create synthetic radar data from the huge, existing databases of video data that are readily available.