IronEar Protects Your Home Without a Camera

Cameras are great for keeping tabs on a home or property, but they come with some obvious drawbacks. They can only see what is in front of them, they do not work particularly well in every environment, and putting cameras everywhere raises privacy concerns of its own. IronEar takes a different approach by listening for important sounds instead, while keeping all of the captured audio on the device.

IronEar is a small sensor node that uses an onboard neural network to recognize more than 500 different types of sounds. Those include things like speech, barking dogs, running water, breaking glass, laughter, and vehicles. Rather than recording audio and uploading it to a server for processing, classification happens locally at about once per second. No recordings need to leave the device, and there is no cloud service or subscription involved.

The hardware is built around an Espressif ESP32-S3-WROOM-1 module with 16 MB of flash and 8 MB of PSRAM. Three Infineon IM73D122 MEMS microphones are arranged in a triangular array to pick up sound. Recognition is handled by TinyEar, a port of Google's YAMNet audio classification model.

IronEar can do more than just recognize predefined sound classes. Its scene learning feature lets the user record the sound characteristics of an environment for a few minutes and then detect that particular acoustic scene later. That provides a way to recognize situations that do not neatly correspond to one of the model's existing classes.

For home use, IronEar connects to Home Assistant through MQTT. A detected sound can be treated much like any other sensor input. Hearing a doorbell could trigger an automation, for example, while the sound of running water could generate an alert. A built-in web interface provides access to live classifications and configuration tools without requiring an app or online account.

When Wi-Fi is unavailable, the device uses a LoRa module based on the Semtech SX1262 to communicate over a private Meshtastic channel. That makes the system useful for off-grid cabins, campsites, farms, wildlife preserves, and other locations where internet access may be unavailable.

A Quectel LC76G GNSS receiver provides each node with data about its position, and a magnetometer provides its heading. When several nodes are scattered across a property, this information can be used to identify which sensor detected activity and where it was located.

Most of the project will be open source, including the hardware design files, firmware, APIs, mesh protocol documentation, and TinyEar model weights. The inference engine and its training data will remain closed. The price and release date have not yet been announced, but you can sign up for notifications on Crowd Supply to stay informed.