Confinement Pandemic Controller (CPA)

The COVID-19 pandemic in Spain is part of the pandemic of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A lockdown was imposed on 14 March 2020.[9] On 29 March, it was announced that, beginning the following day, all non-essential workers were ordered to remain at home for the next 14 days. if you want detail information https://en.wikipedia.org/wiki/COVID-19_pandemic_in_Spain.

Actually there are several restrictions some of them can be controlled thanks to this project and the Openvino Toolkit capabilities.

The core of this project is the OpenVINOtoolkit. Intel says that OpenVINOtoolkit is a toolkit to fast-track development of high-performance computer vision and deep learning into vision applications. And one time that you learn the path is really fast to develop an application.

I have a wifi camera install to control the access to my house but if I move the camera lens I can see a street section.

The objective of this project is to detect cars and people and analyze the behavior in order to check the rules in real time. Sending alarmas and saving images of the violations in order to be review for some human.

The first step is to take a frame from the camera, I use rtsp to access the camera then analyze the frame looking for cars and persons and finally check differents rules and show the reults in a window and send alerts and data using mqtt to a node red server I have installed at home.

The first part and the most important is to detect cars and people. We need an object detection model, Openvino toolkit have several pretrained models. I compared three

person-vehicle-bike-detection-crossroad-0078

Person/Vehicle/Bike detector is based on SSD detection architecture, RMNet backbone, and learnable image downscale block (like person-vehicle-bike-detection-crossroad-0066, but with extra pooling). The model is intended for security surveillance applications and works in a variety of scenes and weather/lighting conditions

https://docs.openvinotoolkit.org/2019_R1/_person_vehicle_bike_detection_crossroad_0078_description_person_vehicle_bike_detection_crossroad_0078.html

person-vehicle-bike-detection-crossroad-1016

MobileNetV2 + SSD-based network is for Person/Vehicle/Bike detection in security surveillance applications. Works in a variety of scenes and weather/lighting conditions.

https://docs.openvinotoolkit.org/2020.1/_models_intel_person_vehicle_bike_detection_crossroad_1016_description_person_vehicle_bike_detection_crossroad_1016.html

pedestrian-and-vehicle-detector-adas-0001

Pedestrian and vehicle detection network based on MobileNet v1.0 + SSD.

https://docs.openvinotoolkit.org/2019_R1/_pedestrian_and_vehicle_detector_adas_0001_description_pedestrian_and_vehicle_detector_adas_0001.html

Comparing the detection rate of the three the last is the best 88% detection pedestrians and 90% vehicles. However after trying different models with my hardware I get the best results with the person-vehicle-bike-detection-crossroad-0078

This the flow for using an Object Detection model.

In our case we use a pretrained model so we need to dowload it using a the python script provide by Intel

>python downloader.py --name pedestrian-and-vehicle-detector-adas-0001

After that you have to use the converter and optimizer

>python converter.py --name person-vehicle-bike-detection-crossroad-0078 --mo 'Path_to_model_optimizer\mo.py' --download_dir 'Path to dowloaded models\deployment_tools\tools\model_downloader\intel'

After that we have two files one with extendion.bin and another with.xml extension in diferents directoris, each for each precission, I use the FP16 model.

As we can see in the model documentation the input of the neural network is: an image of shape: [1x3x384x672] in the format [BxCxHxW], where:

• B - batch size
• C - number of channels
• H - image height
• W - image width.

The expected color order is BGR.

And the output is a blob with shape: [1, 1, N, 7], where N is the number of detected bounding boxes. For each detection, the description has the format: [image_id, label, conf, x_min, y_min, x_max, y_max]

• image_id - ID of the image in the batch
• label - predicted class ID
• conf - confidence for the predicted class
• (x_min, y_min) - coordinates of the top left bounding box corner
• (x_max, y_max) - coordinates of the bottom right bounding box corner.

So we input a frame from the ip camera and get a number of objects. Each object with a label 1 for pedestrian 2 for vehicles, a confidence o prediction probability and the coordinates of a rectangle around the object.

Also we need to know the factor to get the distance in meter when we have a distance in pixels.

The frame is wxh pixels and it corresponds to 15.45m. There is an error because the camera usually is not in front of tha car so the measure has some error.

Using the coordinates we can extract a lot of information from the frame. For vehicles we can measure the number of vehicles, the direction of the vehicles. If we have a parking is easy to know the available capacity. We can measure the speed, the distance between cars in the same direction, the estimate width in order to know if it is a car, a van or a truck. We can see these information for vehicles:

• Vehicle counter: the number of vehicles present in the frame
• Rightcounter: the number of vehicles in the right direction
• Left counter: the number of vehicles in the left direction
• Speed : estimate vehicle speed
• Distance: estimate distance between vehicles in the same direction
• Width: estimate vehicle width

Also we can check if there is a traffic jam or an accident if we see a car o more with an speed of 0 during a time.

We can check if the speed limit, 30kmph in my street, the distance between cars 9m at 30kmph is met. If there is a violation we save a picture of the event and send an alert to node-red, and nodered can send a telegram to alert..

Here we can see some violations:

Distance violation:

Speed violation

For pedestrians we can count the number, the speed if there are more of two the social distancing, the use of the crosswalk.

• Pedestrian counter: number of pedestrians
• Speed: estimate speed
• Distance: distance between pedestrians

If there is a possible violation we save a picture and send an alert. We can see some social distance violation of pedestrians:

Finally we send statistic information and alerts using mqtt to a nodered server. The node red server sen a telegram msg every time an alert is triggered.

In nodered we have a control panel with some statistics, and the alerts received.