A Face Following Robot Car

During my Thanksgiving and winter breaks, I wanted to learn AI with face recognition and computer vision and applying this to a robot. More specifically, I wanted a robot with a camera that would move and rotate itself based on where it sees faces.

I bought a robot chassis kit on Amazon and used the parts that I had to make this car. However, I faced several challenges along the way, and from the project, I learned not only about technical things like how to use the OpenCV library but also a lot about project management.

Videos

How it works

I made this robot with a Raspberry Pi 4 and an Arduino. The camera is attached to the Raspberry Pi, and the Raspberry Pi uses OpenCV to detect the faces in the camera. The Arduino acts as the motor driver. Based on where the faces are, the Raspberry Pi would use the USB port to tell the Arduino how much to move the motors.

However, I still wanted a place to view what the camera is seeing and to manually control the robot, if needed. I chose to do this with a website. The problem with this was that the existing library for communicating using the USB port, pyserial, did not have a convenient way of communication with a web server, like Flask. Because of this and also the fact that I likely have to do something like this for future projects, I created a Python library that starts a web server where a user can interact with the serial port, which is open source on GitHub.

The website I added to see the camera feed and to control the car manually was made with the Vue JavaScript framework. The website includes the video stream that has the bounding boxes of the faces calculated from the backend, and it also adds the ability to control the robot manually using the keyboard (WASD keys) by sending requests to the backend web server created by my library.

For the hardware, I used a robot chassis kit that I found on Amazon, which came with a chassis, 4 wheels, and 4 motors. Then, I used two L298N motor controllers to control all motors. One motor driver was for the left side, and one was for the right side. The motors are powered separately from the Raspberry Pi, with the Raspberry Pi plugged into the wall and the motors powered by 8 AA batteries. All components are hot-glued onto the car.

Challenges

One challenge I faced was wiring the robot. I initially planned to power everything with 8 AA Batteries (12V), and using a voltage regulator to power the Raspberry Pi with 5V. However, the AA batteries did not deliver enough current to power the Raspberry Pi, making it so that it would not boot. To solve this, I had to power the Raspberry Pi and the motors separately, but a 5V power bank still could not deliver enough power, so I had to leave the Raspberry Pi plugged into the wall. Since this robot only rotates and does not drive too much forward or backward, this wasn't ideal but was still somewhat fine.

Another challenge I faced was trying to figure out how OpenCV works and using the right haar cascade file. Some of them could not detect faces when there was low light, and others could not detect faces that were too far away. After some experimentation, I found out that OpenCV's default haar cascade file worked the best in my situation.

One problem that the car currently has is that it over-corrects itself whenever it sees a face. When the motors are supposed to stop, the car's momentum keeps the wheels moving for a short period of time. However, in some cases, this period of time allows the car to overshoot and end up on the other side of the face, making it necessary to move it the other direction. The car would then overshoot in the other direction, and this repeats, which leads to the robot rotating back and forth. This is visible in the demo video. One way I can try solving this if I have a similar problem in future projects is by slowing down motor speed as the robot gets nearer to the face.

What I learned

One important thing I learned was to plan the entire project out, down to the specific details, such as what parts to use, instead of having a broad and general idea and using parts based on what I think I need at the moment. This created problems, such as delays from realizing that I am missing a small but essential part and having to wait a week for it to come before I can continue working on the project. I have never created any project as complex as this one before, and I need to use this skill if I want to make more complex projects, otherwise they may fail or take too long.

Another thing that I learned was to focus on the main goal and not spend too much time working on small details. There were certain times that I spent trying to perfect my COM-Server library, even though those changes were not very important for the main project. I could have used this time instead to work on the car.