Optical Microphone System Recreates Sound From Vibrations
The system features dual cameras and a laser to sense high-speed, low-amplitude surface vibrations that are then used to reconstruct sound.
Engineers from Carnegie Mellon University have developed a novel camera system that can see sound vibrations with a level of precision that it can single out an instrument in an orchestra. Think of it as recreating audio without a microphone. The team from CMU's School of Computer Science's Robotics Institute built the system, which features a pair of cameras and a laser.
According to the engineers, the system can detect high-speed, low-amplitude surface vibrations that the human eye can't perceive. It's those vibrations that can be used to reconstruct sound or audio without a microphone or inference. "We've invented a new way to see sound," explains post-doctoral research associate Mark Sheinin. "It's a new type of camera system, a new imaging device, that is able to see something invisible to the naked eye."
Beyond the cameras and laser, the system utilizes an algorithm that compares speckle patterns captured by a rolling shutter and a global shutter. It uses the differences between the patterns to calculate the vibrations and recreate the audio. A speckle pattern created by the laser refers to the behavior of coherent light in space after it bounces off a rough surface. That behavior changes as the surface vibrate. The rolling shutter rapidly scans an image from one end to the other, while a global shutter captures an entire image at the same time.
The system captures vibrations from objects in motion, such as the movement of strings as a musician strums a guitar, and simultaneously senses individual sounds from multiple points, which are then analyzed and recreated into audible sound. The team's dual-shutter, optical vibration-sensing system could allow sound engineers to monitor the music of individual instruments free from the interference of the rest of the ensemble to fine-tune the overall mix. Manufacturers could also implement the system to monitor the vibrations of individual machines on a factory floor to identify machines that need maintenance.