Airpocket's Semi-tori Meijin Turns Sound Into Vision — and Makes Catching Cicadas a Cinch
Built using low-cost off-the-shelf hardware, this visualization tool creates a heatmap to let you dive into the source of any sound.
Pseudonymous maker "Airpocket" has turned a Sipeed Maix Bit into an audio visualizer with a difference: the Semi-tori Meijin draws the sound as an augmented reality (AR) layer over a live camera feed, showing a heatmap highlighting exactly where it's coming from.
"This device is a very simple portable AR device that allows people who are completely deaf or have difficulty localizing the direction of sound due to hearing impairment in one ear to visually perceive the presence of sound and the position of the sound source," Airpocket explains of the handheld gadget. "The microphone array attached to the front of the device is equipped with seven MEMS microphones, which estimate the direction of the sound source by utilizing the slight time difference it takes for the sound to reach each microphone."
The hardware inside the 3D-printed housing is entirely off-the-shelf, Airpocket notes: a Sipeed Maix Bit development board, which includes an on-board image sensor and enough processing power to handle the visualization task, a Sipeed 6+1 Microphone Array board as the input, an M12-mount lens for the image sensor, and a battery holder with toggle switch for power.
"The microcontroller analyzes the sound information obtained from the microphone array and converts it into a color map, which is then overlaid on the image data captured by the camera and displayed on the back display in real-time," Airpocket explains. "Since the color map of the estimated sound source position and the field of view (FoV) of the camera are significantly different, the camera lens is replaced with a wide-angle lens and the color map is cropped to adjust the display image."
"It can also be used for cicada catching by people with normal hearing," Airpocket notes of the device's potential beyond an aid for the hard of hearing. "With this device, the sound of cicadas can be visualized, so even children who are not good at catching cicadas can always find the location of the cicadas that are chirping. The device was renamed to 'Semi-tori Meijin' in Japanese, which means 'Cicada-catching Master,' because it makes catching cicadas so easy.
A full project write-up is available on Hackaday.io, with source code and housing STL files on GitHub under an unspecified open source license.