Look At Me When I’m Talking to You

Of all the promising wearable electronic devices that have been developed in recent years, earables have proven to be among the most popular. Since we are already accustomed to using headphones and earphones, packing some extra intelligence into these devices is a natural and unobtrusive way to incorporate the power of artificial intelligence (AI) or modern sensing systems into our daily lives. We have recently covered earables that can help the visually-impaired to get around, make us more productive, or even give us more mental focus.

Perhaps one of the most important applications of future wearable technologies will be to help us cut through the noise of modern life. Between constant notifications on our smartphones, a never-ending stream of emails, and noisy, crowded public spaces, how are we supposed to get much of anything done? Focusing our attention is more important than ever, yet it seems harder to do by the day.

Noise-canceling headphones can help to quiet our surroundings to give us more opportunities to focus, but shutting out all sound is not necessarily desirable. Selective noise-canceling is much more useful, as it can allow certain important sounds through the barrier. This is a tough problem to crack, but big strides have been made in recent months.

One of the research groups that has been a leader in selective noise-canceling headphone technologies has now taken their work a step further. The latest iteration of their system can lock in on the voice of a single speaker, canceling out all other noises in the background. This is great news for anyone that has ever tried to have a discussion in a busy public space, only to find that they spend more time repeating themselves than engaging in meaningful conversation.

This system, called Look Once to Hear, leverages an off-the-shelf pair of commercial headphones in conjunction with a custom AI processing pipeline. A pair of microphones, one on each side of the headset, is also included in the build. When the user taps a button, then focuses their gaze on the speaker they want to listen to for a few seconds, the device will capture audio from those microphones.

Since the user is looking directly at the speaker, sound from that individual should reach each microphone at very nearly the same time. Using this information, the algorithm can pick out the voice of the speaker, and train itself to recognize their distinctive patterns of speech. From there, the system is able to filter out all sounds except the speaker’s voice — even if the wearer of the headphones turns their head away after the initial training period.

Look Once to Hear performs its computations on an onboard computer, so it is conceivable that it could be built into a practical consumer-ready device in the future. But there are still a few areas that need to be improved first. At present, the headphones can only lock in on a single person’s voice. And if there happen to be other people speaking loudly in the same direction as the desired target, it will confuse Look Once to Hear.

After the team refines their technology, they hope to build it into earbuds and hearing aids to make it available to a wider audience. But if you can't wait for a commercial release, you can check out the GitHub repository and build your own.