Alexa, Are You Stealing My Passwords?

If you think your smart speaker is only listening in on you after you say the wake word, then you may be in for a surprise. Smart speakers such as Amazon Echo and Google Home are always listening, waiting for their wake words. Moreover, there have been documented cases where up to a minute of audio is transmitted to remote servers for processing without any wake words being present.

As if this were not concerning enough, a research group from the University of Cambridge has shown that it is not only the things you say that can be captured — what you type on your smartphone, like PIN codes and text messages can also be extracted from audio recorded by your ever-present voice assistants.

To test their method, the team created an experimental setup consisting of a Raspberry Pi with a ReSpeaker six-mic circular array. They used this setup to capture audio recordings of volunteers typing on tablets and smartphones. This data was used to train a convolutional neural network and a linear discriminant analysis model to classify taps into numbers and letters. This was accomplished by teaching the models to understand the location on a screen that is being tapped by listening to screen vibrations.

The experiments showed that this exploit is viable, but not yet very mature. Given ten tries, they were able to guess a five digit PIN correctly in 15% of cases. Text reconstruction was able to achieve 50% accuracy. It should also be noted that these results were obtained while ensuring that devices were located no more than half a meter from the microphones, and also that the devices were held in a particular orientation with respect to the microphones.

Notwithstanding these limitations, this is an exploit to keep your eyes on. Newly developed techniques such as this are often refined over time to become more accurate and less limited.