The Hungry Baby Alarm Sends an Alert Before They Begin to Cry

The (loud) problem

Newborn babies are notorious for constantly waking up and crying, and this can occur every two to three hours until they reach two months of age. However, feeding times might be erratic and lead to highly disrupted sleep patterns in the parents, which is why Caleb Olson decided to design an intelligent monitoring system in the hopes of sending an alert before the baby starts to cry.

Identifying hunger with machine learning

To accomplish this feat, Olson started by retrieving the live video feed from his network-enabled baby monitor and used it as the data source for this project. From here, he needed to identify several key behaviors that are strong indicators of hunger, including lip smacking, excessive movements, and bringing the right fist up to its mouth. Detecting these features is achieved through two separate processes. First, an instance of Google's MediaPipe machine learning algorithm takes an image and maps a detailed mesh onto the face. From here, a script measures how frequent lip smacking, among other cues, takes place and only signals a positive indication if it exceeds a predetermined threshold.

Initial programming

All of this data is fed into a single voting system in which each behavior is given a specific weight, with lip smacking and moving a fist towards the mouth being among the most highly rated. The more these actions take place within a short period, the more the baby's hunger meter will fill up, ranging from 0% all the way to 100% likelihood of needing to be fed.

The challenges with pacifiers

One large problem that became apparent soon after testing was that the entire system was rendered ineffective whenever the baby had a pacifier in its mouth, and thus, preventing the MediaPipe algorithm from placing facial tracking points. Getting around this was done by training another machine learning model to recognize whether or not the baby has a pacifier at all. If the model notices the pacifier is missing shortly after detecting it, a strong cue with a weight of 30% is added to the hunger meter. In concert with all of the other cues, this system could determine when hunger is about to strike with a fair bit of accuracy.

An automated feeder?

Now whenever the hunger bar gets near or at 100%, an alert could be automatically generated and sent to Olson's phone just before crying began. As a small joke, he also created the world's first Automated Hungry Baby Feeding System that releases a stream of milk directly into the crib, although it was only tested on his friend for some hilarious results. To see more about how this project was made, you can watch Olson's video here on YouTube.