Hardware components | ||||||
 |
| × | 1 | |||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
 |
| |||||
Facial recognition and identification will soon be playing a major role in our every day lives. The technology opens up a whole new world of possibilities, and applies to almost every aspect of our lives. Use cases of facial recognition/identification include security systems, authentication systems, personalised smart homes, and home care assistants.
What Will We Build?This tutorial will help you to build a Raspberry Pi that allows you to train a Haarcascades model, detect recognised/unknown people, optionally monitor the camera in near realtime via a stream, and communicate with the IoT JumpWay sending sensor and warning messages that will allow your device to autonomously communicate with other IoT devices on your IoT JumpWay network.
This tutorial will use TechBubble Technologies IoT JumpWay Python MQTT Library for communication, OpenCV for computer vision, Motion to stream the webcame on a local port and a secure Nginx server so that the stream can be accessed safely from the outside world.
This example was our original version of TASS, since our move forward with more advanced computer vision libraries and frameworks, we decided to open up the source code.
Python Versions- 2.7
- 3.4 or above
- Jessie
- Linux Motion
- Nginx
- Raspberry Pi.
- Linux Compatible Webcam
There are a few tutorials that you should follow before beginning, especially if it is the first time you have followed any of our Raspberry Pi tutorials or if it is the first time you have used the TechBubble IoT JumpWay Developer Program. If this is the first time you have used the TechBubble IoT JumpWay in your IoT projects, you will require a developer account and some basics to be set up before you can start creating your IoT devices. Visit the following IoT JumpWay Developer Program Docs (5-10 minute read/setup) and check out the guides that take you through registration and setting up your Location Space, Zones, Devices and Applications (About 5 minutes read).
Preparing Your Raspberry PiTake some time to ensure your Raspberry Pi firmware and packages are up to date, and that your device is secure by following the Raspberry Pi 3 Preparation Doc tutorial.
Cloning The RepoYou will need to clone the TechBubble IoT JumpWay Raspberry Pi Examples repository to a location on your Raspberry Pi. Navigate to the directory you would like to download it to and issue the following commands, easiest is to download it to your home directory.
$ git clone https://github.com/TechBubbleTechnologies/IoT-JumpWay-RPI-Examples.git
Next you will need to navigate to the Computer-Vision directory and install the requirements, if you cloned the repository to your home directory, the commands would be as follows:
$ cd IoT-JumpWay-RPI-Examples/Computer-Vision/Python
$ pip install --upgrade pip
$ pip install -r requirements.txt
OpenCv needs to be installed, follow the Installing OpenCV On Your Raspberry Pi tutorial to accomplish this, this is the computer vision library we will be using.
Install Linux MotionWe will use Linux Motion to stream a live feed to a local port on the Raspberry Pi which OpenCv will connect to and read in the frames from the stream. To get Linux Motion set up, follow the Installing Linux Motion On Your Raspberry Pi tutorial.
There are some modifications to make here, in section 9 of the Motion tutorial, it tells you how to modify the directory where the media is saved, for this tutorial you should change that settings to:
/home/YOURUSERNAME/IoT-JumpWay-RPI-Examples/Computer-Vision/Python/media
Don't forget to check out section 10 with regards to turning off the saving of images to save diskspace.
Set Up Domain Name & SSLWe like to make sure that we try to provide tutorials that will help people learn to create secure projects. For the video stream to work securely, you will need to set up a domain name that is pointed to your Raspberry Pi, you will also need to set up an SSL certificate to ensure that the server used for streaming the video is secure. The Setup Domain Name & SSL For Your Raspberry Pi tutorial explains how to do this, if in doubt ask your registrar or host to assist you. If you cloned this repository to your home directory, the paths that you need to use for your CSR and key generation in the following tutorial are:
/etc/nginx/key.key
and
/etc/nginx/csr.csr
Once you have received your signed crt.crt and ca.crt file from certificate authority, you need to upload them to:
/etc/nginx/ca.crt
and
/etc/nginx/crt.crt
We will use Nginx as our server solution and set it up in a way that it has a Grade A+ SSL rating on Qualys SSL Labs SSL Report. To do this we have provided a guide in the Installing Secure Nginx Server For Linux Motion On Raspberry Pi, follow this tutorial to set up your server. You will need of completed the Installing Linux Motion On Your Raspberry Pi and Setup Domain Name & SSL For Your Raspberry Pi tutorials before starting this step.
Securing Your Raspberry Pi With IPTablesThe next security step you should take is setting up IPTables. Follow the Securing Your Raspberry Pi With IPTables document to accomplish this.
Video StreamOnce you have followed the above steps, if they are not already running you need to do the following in this order:
- Start Linux Motion:
$ sudo service motion start
OR:
$ sudo /etc/init.d/motion start
- Start Nginx:
$ sudo service nginx start
OR:
$ sudo /etc/init.d/nginx start
IMPORTANT: This way of streaming is a new feature and we are still ironing out some kinks, if you would like to have OpenCV access the webcam directly and not have Motion/Nginx streaming, uncomment line 43 in TASS.py and comment out lines 44 and 45.
Connection Credentials & Sensor SettingsThe next steps will be to setup your device instance in the IoT JumpWay Developer Console.
- Follow the IoT JumpWay Location Device Doc to set up your device. You will need to set up a device that has a CCTV Camera added via the Sensors/Actuators section.
- Retrieve your connection credentials and update the config.json file with your new connection credentials and camera ID setting (You will need to go into the device page after creating it to get your correct camera ID).
"IoTJumpWaySettings": {
"SystemLocation": 0,
"SystemZone": 0,
"SystemDeviceID": 0,
"SystemDeviceName" : "Your Device Name",
"SystemCameraID":0
}
"IoTJumpWayMQTTSettings": {
"username": "Your MQTT Username",
"password": "Your MQTT Password"
}
Now that the basics are set up, it is time to train your model with your own photos. When you download this repo, there will already be a trained model and processed images in the processed folder, but this model will not recognise you. You should make a good selection of photos of yourself in different positions and lighting. The more photos you train your model on, the more accurate it will be, if your device is not identifying you you simply need to train it with more images of yourself.
You can add as many images as you like (Dependant on the space available on your RPI 3), for as many people as you like. To add training data navigate to the training folder and create a directory, the directory should be a number, and not a number that is already in the processed folder.
Once you have built up your folder of images, head over to TASS.py and change line 34 (self.train = 0) to self.train = 1 and the start the program. The program will loop through your images and if it detects a face it will recreate an image in the format required for the model, save it to a matching folder in the processed directory, and delete the original image to save space. If it does not detect a face it will simply delete the original image as it is useless for the facial recognition.
Once the processing stage has finished, your new model will automatically start training, once training is finished, it will automatically run the main facial recognition program. Put your face in front of your connected webcam and watch the output of the program as it tries to identify who you are.
NOTE: Remove the README file from the processing directory.
Executing The Program$ sudo python/python3 TASS.py
Each time your device detects a person, the device will send sensor data to the TechBubble IoT JumpWay and warning alerts will be sent when the motion sensor picks up an intruder. You can use sensor values and warning messages to trigger autonomous communication with other devices you have connected to your IoT JumpWay Location Space.
On the device edit page, scroll down to the "Create Rules" section under the "Actuators / Sensors". Here you can use the dropdown menu to create rules that allow your device to email you or to autonomously communicate with other devices on its network in the event of status updates, sensor data and warnings.
Each time your device detects a person or an intruder, it will send data to the TechBubble IoT JumpWay. You will be able to access the data in the TechBubble IoT JumpWay Developers Area. Once you have logged into the Developers Area, visit the TechBubble IoT JumpWay Location Devices Page, find your device and then visit the Sensor/Actuator page and the Warnings page to view the data sent from your device.
Please feel free to create issues for bugs and general issues you come across whilst using the IoT JumpWay Raspberry Pi Computer Vision Example. You may also use the issues area to ask for general help whilst using the IoT JumpWay Raspberry Pi Computer Vision Example in your IoT projects.
IoT JumpWay Raspberry Pi Examples ContributorsAdam Milton-Barker, TechBubble Technologies Founder
Comments