Burglar Alarm

Burglar Alarm

Home security cameras are being found in more and more homes these days, with the associated technologies shrinking in both cost and size. People enjoy the convenience of being able to have a peek inside their home any time that they are away to make sure everything is secure and in good order.

There are some issues with many security cameras on the market, however. Those that detect movement, or objects, often send a slew of notifications that are irrelevant — like every time the cat stands up to do a few spins before laying back down. This notification noise has led to a situation where I often ignore the alerts my camera sends me, because I just assume my cat has been on the move again. Obviously this is not a good situation, because it would lead me to miss anything that I really might want to know about.

Aside from inappropriate notifications, there is an even bigger problem. Do you trust that your camera is secure? How sure can you be that no one is snooping on images from inside your home that are being sent to the cloud, whether they are employees of the camera manufacturer, or bad actors that have discovered an exploit? This is especially concerning considering the stories that have broken lately about a major security camera manufacturer that was not exactly up-front about a known vulnerability that existed for years.

To address these issues, I decided to build my own security camera that uses machine learning to detect people, while ignoring irrelevant distractions like pets. All operations are performed entirely on-device to avoid privacy issues. When a person is detected by the camera, the system will send an alert via email.

Hardware

I chose the Renesas RZ/G2L evaluation board kit to power this project. It is very well suited to embedded ML applications, with a dual core Arm Cortex A55 processor (1.2 GHz), and an Arm Cortex M3 processor (200 MHz) on-board. The 2 GB of SDRAM and 512 MB of flash memory were well in excess of what I needed to make my project work. Gigabit ethernet made it simple to get connected to the Internet to send email notifications. A USB webcam was used to capture images, and was immediately detected by the RZ/G2L, so I didn't have to worry about installing any drivers.

To get the board up and running, I built the latest board support package (BSP) with the core-image-weston option for graphics support, and the SDK. I wrote the BSP to an SD card and popped it in the board, after which it booted into Yocto Linux. After installing the SDK, I could cross-compile C/C++ programs on my Ubuntu laptop that will run on the RZ/G2L board.

Machine Learning

With the hardware ready to go, it was time to build an application that captures webcam images, determines if people are in the frame, and if so, sends an email alert. I used OpenCV to capture webcam images and display them on the screen, and chose Edge Impulse to simplify building a machine learning object detection pipeline that is optimized for embedded devices.

Data Collection

To build an object detection pipeline, you first need examples of the objects that you want to detect, so I wrote a program to collect webcam images with and without people present in them. In total, I captured 461 images such as this:

I uploaded the images to Edge Impulse, then used the data acquisition tools to add bounding boxes to the type of object I want to detect (i.e.: person):

Object Detection

To build the model, I created an impulse consisting of a preprocessing step to resize the input images for more efficient downstream analysis. Next, an object detection neural network was added. This neural network implements Faster Objects, More Objects (FOMO), which was designed for real-time object detection on resource-constrained devices. It is 30x faster than MobileNet SSD, yet runs with <200K of RAM.

I trained the model, and it was found to have achieved a 99.1% object classification accuracy rate:

Deploying the Model

Edge Impulse offers a number of options to deploy your model such that it can run natively on the physical hardware. In this case, the best option was to export the model as a C++ library. There is some solid documentation to help get the library deployed to the hardware. In summary, you can clone the standalone inferencing example project, copy-and-paste the files generated by the deployment process in, then modify the source/main.cpp file to fit your project.

For the RZ/G2L board, I cross-compiled this code on my laptop, then transferred it over to the board via scp.

Result

I have created a demo video to show the security camera in action. Notice the on-screen "Intruder Alert" message when I walk into the frame.

Images are continually captured via the webcam, then displayed on a monitor (via the RZ/G2L HDMI output). Each image is fed into the Edge Impulse object detection pipeline to check for the presence of a person. If a person is found, an "Intruder Alert" message is displayed on-screen, and an email is sent, via a Python script, to a specified email address.

Conclusion

By keeping all of the image captures and processing local, privacy concerns have been eliminated. There is no unknown handling of the data at remote locations by a 3rd party. Further, by detecting specifically the presence of a person (as opposed to a pet, or flash of bright light through a window from a passing car), notifications are only sent when they are relevant and worthy of attention.

I have made my Edge Impulse project public. Check it out here.