PrivacyMic, the Smart Speaker That Doesn't Eavesdrop
UM researchers developed a system that enables devices to understand what's happening in your home without listening to your conversations.
A team of University of Michigan researchers has developed a prototype system dedicated to privacy: a smart speaker that doesn't pick up or record audible sound but still informs a smart home or listens for the signal that would turn on a smart speaker. With an estimated 320 million microphones worldwide always listening to pick out our commands, this is an important step in revamping smart tech to protect user privacy; with the PrivacyMic, smart systems can understand what is happening in your home respond without listening to your conversations.
The PrivacyMic works by picking up only ultrasonic sound at frequencies above the range audible to humans and piecing together these bits of information to identify when its services are needed. These ultrasonic soundwaves carry a great deal of information; for example, while CFL lightbulbs have been engineered not to hum audible, this just means the hum has been tuned up into the ultrasonic range. Due to this, the PrivacyMic can tell things like when the lights are on, when the microwave is on, or when the water is running without needing microphones to record audio. Machine learning is used to classify the ultrasonic sounds and infer what is happening in the microphone’s surroundings. The team collected sound from 127 different common objects, then compressed the ultrasonic signature containing key bits of information and built a Raspberry Pi-based device to listen for them. Demonstrations have shown it to be able to identify household and office activities with greater than 95% accuracy.
An audio filter included on the devices means that all speech and audible sound is filtered outright at that point, making it more secure than encrypting the audio data or other security measures to limit who has access to it. Instead, no sensitive information is stored. At its current stage, the PrivacyMic is a proof of concept, but implementing similar technology in a device like a smart speaker would only take a little tweaking. And while smart speakers are the obvious application, the researchers envisions other uses — like monitoring the homes of the elderly for signs they need help or monitoring lung function in respiratory patients — that could be even more crucial.
The research can be read as it was presented at the ACM CHI Conference on Human Factors in Computing Systems. Of the device, Alanson Sample, U-M associate professor of electrical engineering and computer science, says, "Smart technology today is an all-or-nothing proposition. You can either have nothing or you can have a device that's capable of constant audio recording. PrivacyMic offers another layer of privacy — you can interact with your device using audio if you choose or you can have another setting where the device can glean information without picking up audio."