Robin Sylvester
Published © GPL3+

Dynamic Face Recognition based entry and exit system

Grants access to authorized personnel and stores their entry and exit time. Alerts the security in case of unauthorized personnel.

IntermediateFull instructions provided5 hours2,230
Dynamic Face Recognition based entry and exit system

Things used in this project

Hardware components

Bolt WiFi Module
Bolt IoT Bolt WiFi Module
×1
LED (generic)
LED (generic)
Used as a substitute for electromagnet. The LED acts as a door status indicator.
×1
Resistor 330 ohm
Resistor 330 ohm
×1
Buzzer
Buzzer
×1
Breadboard (generic)
Breadboard (generic)
×1
USB-A to Mini-USB Cable
USB-A to Mini-USB Cable
×1
Male to Male breadboard wires
×1
Male to Female wires
×3
Camera (generic)
Separate cameras for entry and exit
×2
Speaker
To voice out the name of the entering person
×1

Software apps and online services

Bolt IoT Android App
Bolt IoT Android App
Windows 10
Microsoft Windows 10
sql server

Story

Read more

Schematics

Face Registration

This is a schematic representation of the face registration process.

Authentication Control

This is a schematic representation of the authentication process. Instead of the electromagnet in the diagram, I have used LED for the demo.

Real Time Face Recognition

This diagram is an overview of the entire system

Code

Face Registration

Python
Stores the names and face encoding of the given images and generates the welcome audio for every name and saves it. Add your own images to a folder and specify its path in the face registration code.
import face_recognition
from gtts import gTTS



# Load a sample picture and learn how to recognize it.
robin_image = face_recognition.load_image_file("robin.jpg") # specify the path of your image
robin_face_encoding = face_recognition.face_encodings(robin_image)[0]

# Load a second sample picture and learn how to recognize it.

eswar_image = face_recognition.load_image_file("eswar.jpg") # specify the path of your image
eswar_face_encoding = face_recognition.face_encodings(eswar_image)[0]



# Create arrays of known face encodings and their names
known_face_encodings = [
    robin_face_encoding,
    eswar_face_encoding

]

# Enter the names of the specified images
known_face_names = [   
    "Robin Sylvester",
    "Eswar"

]


# Generate 'Welcome' audio for all the known face names.
known_face_names_length = len(known_face_names)
for i in range(known_face_names_length):

   tts = gTTS("Welcome"+known_face_names[i], lang='en')
   tts.save('_'+str(i)+'.mp3')

Entry System

Python
Code for entry system. Don't forget to enter your Bolt API key and device ID in the entry and exit code and also enter your driver, server and database name in the entry and exit code.
import face_recognition
import cv2
from pygame import mixer
import time
from datetime import datetime
from face_reg import known_face_encodings
from face_reg import known_face_names
from boltiot import Bolt
import pyodbc
import itertools


#Bolt iot
api_key = " "  # Enter your Bolt API Key
device_id  = " " # Enter your Bolt  device ID
mybolt = Bolt(api_key, device_id)


# Database Connectivity
conn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};'   # Enter your driver, server and 
                      'Server=DESKTOP-TDBKSHJ\SQLEXPRESS;'        # and database name
                      'Database=face_rec;'
                      'Trusted_Connection=yes;')
cursor = conn.cursor()

# '0' for primary camera and '1' for secondary camera
video_capture = cv2.VideoCapture(0)

# Initialize some variables
face_locations = []
face_encodings = []
face_names = []
seen_faces = []
process_this_frame = True

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resizing frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:

        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding,0.5)
            name = "Unknown"

            if True in matches:
                first_match_index = matches.index(True)
                name = known_face_names[first_match_index]

                # Get the indices of seen faces from the database and store it in the seen_faces list
                cursor.execute('select seen_face_index from dbo.seen_face')
                result = cursor.fetchall()
                seen_faces = list(itertools.chain(*result))

                # The condition for entry is either the seen_faces list should be empty or the matched index should
                # not be in the seen_faces list
                if len(seen_faces) == 0 or first_match_index not in seen_faces:
                   mixer.init()
                   mixer.music.load('D:\Projects\MP\pypi\_'+ str(first_match_index) + '.mp3')
                   # Play the welcome audio
                   mixer.music.play()

                   # insert the matched index into the column of seen face indices
                   cursor.execute('insert into face_rec.dbo.seen_face(seen_face_index) values(?)',
                                  (first_match_index))
                   conn.commit()

                   # Get the current date and time
                   now = datetime.now()
                   dt_string = now.date()
                   tm_string = now.time()

                   # insert the time and date to the 'entry_time' table in the data base
                   cursor.execute('insert into face_rec.dbo.entry_time(emp_id,indate,intime) values(?,?,?)',(first_match_index,dt_string,tm_string))
                   conn.commit()

                   #  Door Control
                   response = mybolt.digitalWrite('0', 'HIGH')
                   # number of seconds the door should be left open
                   time.sleep(2)
                   response = mybolt.digitalWrite('0', 'LOW')

            # alert the security
            if(name=="Unknown"):
                response = mybolt.digitalWrite('1','HIGH')

            # add the name to the face_names list
            face_names.append(name)


    process_this_frame = not process_this_frame

    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
       break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

Exit System

Python
Code for exit system. Don't forget to enter your Bolt API key and device ID in the entry and exit code and also enter your driver, server and database name in the entry and exit code.
import face_recognition
import cv2
import time
from datetime import datetime
from face_reg import known_face_encodings
from face_reg import known_face_names
from boltiot import Bolt
import pyodbc
import itertools

# Bolt iot
api_key = " " # Enter your API Key
device_id = " " # Enter your device ID
mybolt = Bolt(api_key, device_id)

# Database Connectivity
conn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};' # Enter your Driver, server and
                      'Server=DESKTOP-TDBKSHJ\SQLEXPRESS;'      # database name
                      'Database=face_rec;'
                      'Trusted_Connection=yes;')
cursor = conn.cursor()

# '0' for primary camera and '1' for secondary camera
video_capture = cv2.VideoCapture(1)

# Initialize some variables
face_locations = []
face_encodings = []
face_names = []
seen_faces = []

process_this_frame = True

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resizing frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:

        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding, 0.5)
            name = "Unknown"

            if True in matches:
                first_match_index = matches.index(True)
                name = known_face_names[first_match_index]

                # Get the indices of seen faces from the database and store it in the seen_faces list
                cursor.execute('select seen_face_index from dbo.seen_face')
                result = cursor.fetchall()
                seen_faces = list(itertools.chain(*result))

                # person is allowed in the exit gate only if the seen_faces list is not empty or matched index
                # is in the seen faces list
                if len(seen_faces) != 0 or first_match_index in seen_faces:

                    # remove the matched index from the 'seen_face' table in the database
                    cursor.execute('delete from dbo.seen_face where seen_face_index=?',
                                   (first_match_index))
                    conn.commit()
                    print(seen_faces)

                    # Get the current date and time
                    now = datetime.now()
                    dt_string = now.date()
                    tm_string = now.time()
                    cursor.execute('insert into face_rec.dbo.exit_time(emp_id,out_date,out_time) values(?,?,?)',
                                   (first_match_index, dt_string, tm_string))
                    conn.commit()

                    #  Door Control
                    response = mybolt.digitalWrite('0', 'HIGH')
                    # number of seconds the door should be left open
                    time.sleep(2)
                    response = mybolt.digitalWrite('0', 'LOW')


            # alert security
            if (name == "Unknown"):
                response = mybolt.digitalWrite('1', 'HIGH')

            face_names.append(name)


    process_this_frame = not process_this_frame

    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

Security's access control

HTML
It is the script for the product linked with the bolt device. This script helps the security to manually access the door and buzzer.
<!DOCTYPE html>
<html>
    <head>
        <title>Bolt IoT Platform</title>
        <script type="text/javascript" src="https://cloud.boltiot.com/static/js/boltCommands.js"></script>
        <script>
        setKey('{{ApiKey}}','{{Name}}');
        </script>
    </head>
    <body>
        <center>
            <h1>Door Control</h1>
        <button onclick="digitalWrite(0, 'HIGH');">Open</button>
        <button onclick="digitalWrite(0, 'LOW');">Close</button>
        </center>
        <br>
        <hr>
         <center>
            <h1>Buzzer Control</h1>
        <button onclick="digitalWrite(1, 'HIGH');">ON</button>
        <button onclick="digitalWrite(1, 'LOW');">OFF</button>
        </center>     
    </body>
</html>

Credits

Robin Sylvester

Robin Sylvester

1 project • 4 followers

Comments