Iwatake's TinyML "Loud Talking Detector" Uses a Raspberry Pi Pico to Shame Restaurant Shouters
Device could "encourage people in restaurants/cafes to eat quietly," its creator explains, "to prevent the spread of the coronavirus."
Pseudonymous engineer Iwatake is continuing a series of experiments with the Raspberry Pi Pico, this time turning the microcontroller board into a device for detecting — and shaming — people who talk too loudly in restaurants, using TensorFlow Lite.
"This tinyML system uses a Raspberry Pi Pico and TensorFlow Lite for Microcontrollers to detect loud talking," Iwatake writes by way of introduction to the project. "It can be utilized to encourage people in restaurants/cafes to eat quietly to prevent the spread of the coronavirus and help in the fight against COVID."
"It detects 'talking' when people talk loudly. It doesn't detect 'talking' when people talk quietly or the sound is not talking (e.g. noise, music, etc.) A deep learning model is created to classify 10 or 5 seconds of audio to two types of sound ('Talking,' 'Not Talking'). The model is converted to TensorFlow Lite for Microcontrollers format. The model is deployed to a Raspberry Pi Pico."
The model, which was trained using Google Colaboratory, runs on the Raspberry Pi Pico with a display and a microphone attached. Sound is captured by the microphone, processed through the model, and the result output to the display — all in a device which lives happily in a small mint tin.
"This is a very tiny system (fitting in FRISK [mint tin]!), so that it can be implemented in an order call system in restaurants to encourage customers to eat quietly to prevent the spread of the coronavirus," Iwatake explains. "[I] need to reduce power consumption [and] improve accuracy. So far, the training data is very limited (most of them are Japanese)."
The project isn't Iwatake's first shot at working with the Raspberry Pi Pico: Earlier this year a project was unveiled which used a microphone and a Raspberry Pi Pico to display a live-view audio spectrum analyzer with FFT, while prior to that Iwatake had shown off a TensorFlow Lite project to give the Raspberry Pi Pico the ability to recognize hand-written numbers.
Source code, under the permissive Apache 2.0 license, and documentation for the project is available on Iwatake's GitHub repository.