In large lecture halls in many universities, it is difficult if not impossible to take attendance, leaving professors to either resort to using annoying online forms or clickers which can easily be manipulated or to not taking attendance at all, which results in poor attendance and loss of learning. A device that can perform facial recognition is perfect for solving this problem.
With the NVIDIA Jetson Nano Developer Kit, this project can be made to have a low profile on the ceiling or doorway of any lecture hall, scanning faces and taking attendance as lecture occurs without causing any disruption to class. This is different from existing solutions because the current attendance systems all require input from either the professor or students. In addition, there are very few solutions that are accurate to make sure nobody is gaming the system. It is extremely useful because class attendance is necessary for learning, but taking attendance takes away time for learning, so this system allows learning to happen to its fullest potential. The Jetson Nano Developer Kit is powerful enough to do all this processing on the board itself, requiring no internet connection. It also is compact enough to not be a distraction. The main features of this are to take attendance and learn different appearances so teachers and students do not need to take class time to take attendance.
To build this project, I first set up the Jetson Nano using Nvidia's provided guide here: https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit. This uses the NVIDIA JetPack SDK to include CUDA libraries for hardware-accelerated vision processing.
Then, I plugged in a keyboard, mouse, monitor, and UVC-compatible USB webcam to be able to use the Jetson Nano.
You may also use a camera that plugs into the MIPI CSI camera connector. The code (in this case) is the same.
When you plug in the Micro-USB power cable to the Jetson Nano, it will automatically boot up, and you will go through the standard Ubuntu new user setup process.Installing the Required Libraries
Python 3.6 and OpenCV are already installed with the Jetson Nano version of Ubuntu. However, to run vision processing we need a few more.
Type the following commands into your terminal:
sudo apt-get update
sudo apt-get install python3-pip cmake libopenblas-dev liblapack-dev libjpeg-dev
# These next two allow the Jetson to use storage space as RAM.
git clone https://github.com/JetsonHacksNano/installSwapfile
pip3 install numpy
tar jxvf dlib-19.17.tar.bz2
# scroll to line 854 and comment out the line. It should now read: //forward_algo = forward_best_algo;
# save the file, then return to your terminal and type the following. It will take about 30 minutes.
sudo python3 setup.py install
# now we need to install a custom library for facial recognition that makes our code much simpler!
sudo pip3 install face_recognition
After installing all of these libraries, you are ready to use your Jetson Nano for face recognition!
Writing the Code
See my code file below called main.py. It is commented to allow you to better understand the code.
In short, the code file uses OpenCV to capture video from the webcam, then uses linear algebra techniques to match the face it sees to one in the list of known faces. Then, once it matches a face, it will add the name and email to a text file.
The next file, called helper.py, takes the names and emails from the text file and sends an email using SMTP to each email in the file. This way, students will receive an email confirmation that they have been detected walking into the door.
After setup, the system can be run with only power, the webcam, and Ethernet plugged in. The HDMI, mouse, and keyboard can be unplugged. An optional USB wifi dongle can be added in place of the Ethernet cable.
If internet is not accessible, the list of students is saved locally onto the Jetson Nano's SD card and can be used to track attendance without internet access.
In the future, I hope to implement a deep learning model into this software, so that the software can detect faces more accurately, if a student has changed their appearance. This way there will be fewer false positives and negatives and the system will be more accurate. As is, however, it definitely will achieve its goal of saving professors and students time.
Note: Thanks to @ageitgey on Medium and Github for providing a guide on which I based my project. Used under the MIT License.