Hardware Accelerated Security System from Digilent Contest

In last years, the algorithms complexity and video resolution has increased. The cost of a powerful computer which can run image recognition software is getting bigger as video cards prices go up. Alternatives must to be found, big companies started developing hardware components for specific tasks in order to hardware accelerate the algorithm. We developed a solution which accelerates image processing, like facial recognition, in hardware. The system is much cheaper and consumes less energy, than a regular PC and it is more powerful.

Objectives

The aim of the project was to develop a system which can run applications that are usually run on powerful machines, on much slower and cheaper ones using hardware acceleration.

Features-in-Brief

Hardware accelerated facial recognition

Voice assistance integration

Live streaming

Sends notifications on phone

Allows to open door using a pin

Temperature monitoring

Remote access to your system using a webserver

Project Summary

A hardware-accelerated security system that runs facial recognition in hardware, takes photos when motion is detected, allows you to unlock the door using facial recognition or a pin and even sends a notification on your phone.

Detailed Software Architecture

Main controller

It coordinates everything in the system, it doesn’t need a lot of resources.

Temperature reading

Reads the temperature from the custom IP and save it into a file to be used by web server.

Keyboard module

Check if there are any key presses. It also stores the last 4 keys to check the inserted pin. It the pin is correct, it unlocks the door.

Door Control

Checks if it receives any commands from server, a familiar face was detected or a correct pin was inserted. If one of the above conditions are met, it unlocks the door for few seconds.

Light Control

Checks if it receives any command from the web server to change the state of light.

Motion detector

Gets a frame from camera and writes it into memory. Then, it sends it to the motion detector to check for any pixel changed. If the sum of all modified pixels is bigger than threshold, it starts the face detect part.

Face detect

Get from hardware component the location of faces in image and sends them to facial recognition algorithm developed in python.

Live stream

Stream the swf live video stream from webcam to a remote location on port 8081.

Notification System

The system can send notifications to your phone. The phone checks periodically the web server for any events in the system. If the door was unlocked, the user receives a notification on the phone.

Web server

The Zybo Z20 also hosts a web server using integrated 1Gbs ethernet port.

The Web server is created on Linux using apache, it also has a database for saving room logs and pictures when detecting motion.

Voice assistance

The voice assistance is not running on Zybo, we are planning to hardware accelerate this one too, but for now, it is running on Amazon Alexa Dot with a custom skill wrote in C# for it.

Facial recognition

The facial recognition is running the dlib library. It is the state of the art in facial recognition. The library uses deep metric learning tooling and the accuracy of algorithm is 99.38% based on Labeled Faces in the Wild benchmark.

Linux module

A Linux module for Kernel was also developed to allocate the memory and convert the virtual address to physical. The DMA uses physical addresses and the Controller is dealing with virtual addresses. It allocate the size of a Full HD frame and sends to Controller App the virtual address to access it and the physical address to be able to transfer it to the DMA.