StoryIntroductionFor our ELEC 424 final project, we programmed an RC car to drive itself. The design itself is simple; it uses a Raspberry Pi 5, a USB webcam, and a small wheel encoder to follow a blue tape track on the floor of PCF3 @ Rice, stop at two red boxes along the way, and know when it's reached the end.
As a challenge, although undergraduates, we also got YOLOv5 running on the Pi 5, so the car can identify objects in real time through the webcam.
To start, we built on User raja_961's Instructable, 'Autonomous Lane-Keeping Car Using Raspberry Pi and OpenCV' (but had to rewrite a lot to get it working on the Pi 5: swapping out the old GPIO library for lgpio and building a custom kernel driver for the encoder using gpiod).
https://www.instructables.com/Autonomous-Lane-Keeping-Car-Using-Raspberry-Pi-and/
Resolution and PID TuningIn the end, we selected a capture resolution of 160×120 because it minimizes processing latency, allowing the control loop to consistently run above 25 FPS while still providing enough tape width for reliable centroid detection.
For the steering controller, we empirically tuned the proportional gain to Kp = 0.020 to provide sufficient cornering authority, and introduced a small derivative gain of Kd = 0.004 to dampen oscillations and smooth the vehicle's turn-in response.
# ── PD controller gains ───────────────────────────────────────────────────────
# error = (tape_x - frame_center_x), in pixels. Frame is 160px wide, so max error ≈ ±80.
# steering output = Kp*error + Kd*d(error)/dt, clamped to [-1, 1].
#
# Kp = 0.020 → full servo deflection at ~50px error (good for tight turns).
# Kd = 0.0 → derivative term disabled. When enabled (e.g. 0.004), it amplifies
# frame-to-frame noise in the centroid position into rapid servo twitching.
# Leave at 0 unless you have a very stable, low-noise detection signal.
Kp_steer = 0.020
Kd_steer = 0.004
prev_steer_error = 0
STEER_TRIM = 0.0 # add a small offset (e.g. 0.05) if the car drifts left or right on a straight
An integral gain was deemed unnecessary; the vehicle exhibited no persistent mechanical steering bias (allowing our hardware steering trim to remain at 0.0), meaning there was no steady-state error for an I-term to correct.
Stop BoxesThe red stop boxes are detected using a two-range HSV mask (red wraps around 0° in HSV, so we need both [0, 100, 100]–[10, 255, 255] and [160, 100, 100]–[180, 255, 255]) followed by cv2.findContours and an area threshold of 1000 px² to reject small red specks like tape edges or background noise.
# ── Red stop-box HSV range ────────────────────────────────────────────────────
# Red wraps around 0° in HSV, so we need two separate ranges to catch it.
# Lower range: 0-10 (red on the low end), Upper range: 160-180 (red on the high end).
RED_LOWER1 = np.array([0, 100, 100], dtype="uint8")
RED_UPPER1 = np.array([10, 255, 255], dtype="uint8")
RED_LOWER2 = np.array([160, 100, 100], dtype="uint8")
RED_UPPER2 = np.array([180, 255, 255], dtype="uint8")
RED_AREA_THRESHOLD = 1000 # minimum contour area (px²) to count as a real red box, not a red pixel speck
To keep the lane-keeping loop responsive, we only run red-box detection on every 3rd frame. The car uses a small state machine:
DRIVING → STOP_CD → STOP1 → DRIVING → STOP_CD → STOP2.
The STOP_CD state exists because a single noisy frame should never trigger a stop. Instead, we count down 10 frames after the initial detection before committing to the stop, which eliminates false positives caused by transient red glints.
After the first stop, we hold neutral throttle for 3 seconds and then add a 4-second cooldown before the red detector can re-arm, preventing the same stop box from triggering twice. After the second detected box, the state machine enters STOP2, and neutral() is called on every frame thereafter.
Plot 1: Error, Steering Duty Cycle, and Speed Duty Cycle vs. Frame Number
Plot 2: Error, Proportional Response, and Derivative Response vs. Frame Number
Image of Embedded Car Hardware
Video of Vehicle Completing the Course:
Video of ObjectDetection:
Comments