Step Up Your AI with a Pi 4 on the Floor
Do footsteps have fingerprints? Adversarial learning on a Raspberry Pi can uniquely identify people by the sound they make while walking.
When building personalized services, healthcare monitoring devices, or security applications, there is a common thread — the need for user identification. Most commonly, passive identification is handled with computer vision solutions or biometric sensors. These techniques have proven themselves very effective, but do have some drawbacks. Camera-based solutions often feel invasive, and in many contexts are not acceptable to users. Biometric sensors may not give the same uncomfortable feeling of being watched, but they typically require that a device be worn, which can be cumbersome.
A new passive, multi-person identification system has recently been developed by researchers at Nanyang Technological University that — you might say — sidesteps these issues. Named PURE (Passive mUlti-peRson idEntification), the method can detect individuals by the sound of their footsteps. Impressively, the team has been able to identify individuals with audio data from as little as a single footstep.
The PURE hardware consists of a circular microphone array with a sampling rate of 192 kHz, and a Raspberry Pi 4. The signal processing and machine learning algorithms, implemented in custom C++ code and Tensorflow, run on the Raspberry Pi.
The pipeline first subtracts background noises, such as voices, that would otherwise overwhelm the more subtle signal acquired from footsteps. The user identifying neural network was trained via an adversarial domain adaptation scheme to ensure good generalization.
Recognizing that an attacker could record the sound of someone’s footsteps and replay them later to trick the system, the researchers built in a safeguard. Observing that replayed sounds exhibit static spatial characteristics, they incorporated the smoothly changing spatial characteristics of captured footstep sounds into a replay defense mechanism.
In a series of one hundred trials, PURE was found to accurately determine the identity of the individual from the sound of their footsteps up to 88.6% of the time from a single footstep. In the case of two or three footsteps, accuracy up to 95.92% and 96.53% was achieved, respectively.
These days, it seems as though a new person identification technique is developed each week. Whether or not you consider that a good thing is a matter of opinion. In any case, thirty-seven years after one of the great security researchers of our era first expressed his concerns, I believe we now conclusively have a clear answer...yes, Rockwell, it’s true. Somebody’s always watchin’ you.