Raspberry Pi RC Car with Learned Steering

OVERVIEW

I built this Raspberry Pi RC car to explore how a small neural network can learn steering from camera images. The project brings together data collection, supervised learning, model conversion, and real-time control on embedded hardware.

The current version provides learned steering. Throttle stays under manual gamepad control. It does not provide obstacle avoidance or fully autonomous navigation.

HOW IT WORKS

During manual driving, the camera records images while the controller supplies steering and throttle labels. A PilotNet-style convolutional neural network learns to predict steering from those images. Training runs on a computer. The trained model is exported to TensorFlow Lite format and runs on the Raspberry Pi through ai-edge-litert.

At runtime, the Pi captures an image, applies the same preprocessing used during training, predicts a steering value, smooths it, and commands the steering servo. The operator controls forward and reverse throttle using the gamepad triggers.

HARDWARE

The build uses a Raspberry Pi, a Picamera2-compatible Pi camera, an RC chassis with a drive motor and steering servo, an appropriate motor driver, and a pygame-compatible gamepad. The motor needs a suitable separate power supply, with a common ground shared with the Pi and driver. A computer is used to train the model.

GPIO SIGNAL ASSIGNMENTS

The current repository uses BCM numbering:

Steering servo signal: GPIO 18, with a configured pulse range of 1300–1900 microseconds.

Motor driver forward input: GPIO 5.

Motor driver backward input: GPIO 6.

Motor driver enable/PWM input: GPIO 19.

These assignments describe control signals, not a complete power schematic. Driver terminal names depend on the actual driver. Do not power the motor from Pi GPIO pins.

1. COLLECT DRIVING DATA

On a compatible Raspberry Pi OS installation, enable the camera and install the dependencies listed in the repository. The code uses PiGPIOFactory and needs the pigpiod service running.

Run data_collection.py and drive manually along the intended track. Each session creates a run directory containing JPEG frames and labels.csv. The CSV stores frame, timestamp, steering, throttle, smooth_steer, and smooth_throttle. Recording is toggled by controller button index 2. Confirm this mapping on your own controller.

Capture examples of left turns, right turns, and straight driving in varied lighting. Keep complete recording sessions separate when making the training and validation sets; nearby frames from the same session are too similar to provide a strong independent check.

2. PREPROCESS AND TRAIN

Transfer the complete recording runs to the training computer. Use test_crop.py to inspect the crop and configure params.py. Current repository defaults crop image rows 60:140 and resize to 68 x 68 x 3. The model rescales pixel values internally.

Run train.py. The training setup includes brightness and blur augmentation, horizontal flips with the steering sign reversed, AdamW, weighted Huber loss, early stopping, and a held-out validation run. The default target is smooth_steer. Training writes H5 and TFLite models.

3. VALIDATE BEFORE DRIVING

Run validation.py on a recording excluded from training. It reports raw and smoothed mean squared error, correlation, and separate errors for left, right, and straight driving. It also plots predictions against recorded steering labels.

The repository includes automated checks for preprocessing, validation-run exclusion, and per-sample loss weighting. Those tests do not exercise the camera, GPIO, or physical driving. There are no published reproducible road-performance or FPS benchmarks in the repository, so this page does not claim an accuracy or speed result.

4. RUN ON THE RASPBERRY PI

Copy the trained TFLite model to the Pi alongside params.py and preprocessing.py, then run autonomous_drive.py. Keep the camera-to-file color-channel convention consistent between recording, training, and inference. The current inference path expects the floating-point model exported by the training flow.

The script smooths and clamps predicted steering to the range -1 to 1. Manual throttle is capped at 0.4 in the current scripts. Controller button index 0 stops driving; verify button mappings before use.

PRACTICAL CHECKS AND LIMITATIONS

Test steering direction and motor control with the wheels lifted first, then start slowly in a clear, controlled area with an operator at the controls. The current code has no obstacle detection or hardware-disconnect watchdog. Changes to the camera, crop, lighting, or smoothing require a new evaluation. Previously trained weights should be reevaluated after preprocessing or loss changes.

WHAT I LEARNED

This project connects the full edge-AI workflow: gathering labeled data, preventing validation leakage, keeping preprocessing consistent, training a compact model, and connecting inference to physical actuators. It also shows why software tests and real hardware evaluation answer different questions.

SOURCE CODE AND REPRODUCTION

The complete scripts, dependency files, wiring table, and setup commands are available here:

https://github.com/coder-og/RC-Car-Autonomous-end-to-end-tinyML

Start with README.md. The main flow is data_collection.py, test_crop.py, train.py, validation.py, and autonomous_drive.py. A trained model and dataset are not bundled; collect your own data and train before running the car.

BUILD PHOTOS FROM THE PROJECT REPORT

These photographs are extracted from page 22 of the original report and show the physical build documented there.

Top view: chassis, electronics, and wiring.