Autonomous Driving Car V2

This project is all about building a cool self-driving car using some fun and exciting technology! We'll be using a Raspberry Pi as the driver of the car to make all the decisions and see the world through a camera. We'll also be using a XMC microcontroller board to control the car's steering and speed. To make the car move, we'll be using a Motix DC motor shield that sits on top of the XMC boot kit.

Donkey Car

Donkey Car is an open-source platform for building and experimenting with self-driving cars. It provides an easy-to-use software framework that allows users to quickly and easily train and deploy machine learning models on a small, inexpensive hardware platform, such as the Raspberry Pi.

The platform includes a variety of pre-built software components for tasks such as image processing, control, and communication, making it easy for users to get started with building their own self-driving car. Additionally, the community provides a lot of resources such as tutorials, documentation, and sample code, which can help users to understand the inner workings of the platform and learn how to build their own projects.

The Donkey Car project runs on the Raspberry Pi and acts like the driver.You collect datasets while driving your car remote controlled. This datasets are then used to train a neural network to drive the car on its own.

Wait machines can actually learn??

A neural network is like a computer brain that can learn to do different things like recognizing pictures or controlling a car. It's made of many small parts called neurons that work together to take in information and make decisions or predictions. We can train a neural network by giving it examples of what we want it to do, and it will adjust itself to do so (it learns). Neuronal networks are used in different fields like image recognition, speech recognition, natural language processing and self-driving cars. In summary, a neural network is a computer program that can learn to do different tasks by analyzing data and examples.

Project setup

The Raspberry Pi takes in power from a 5V power supply. (Any power bank should do the trick here ;)). The camera connected to it records the surrounding environment and the pictures are fed into a neural network which determines what the next moves of the car should be.

The communication between Raspberry Pi and XMC microcontroller is based on I2C.

The XMC is controlling the direction and speed of the DC Motor as well as the servo position in charge of steering. The drivetain motor is controlled using a BTN99x0 shield (see the shield's protip to know more about how this happens by clicking here). The whole stack is powered by a 2S Lipo Battery.

Servo Motors

A servo motor is a type of motor that can be controlled to rotate to a specific angle. To use a servo motor, you will need to connect it to a microcontroller, such as the XMC which sends pulse-width modulated (PWM) signals to control the position of the servo's shaft. The pulse width of the PWM signal determines the position of the shaft, with different widths corresponding to different angles.

Crash Course I2C

I2C (Inter-Integrated Circuit) is a communication protocol that allows multiple devices to communicate with each other. It is a master-slave protocol where one device controls the communication and the other devices respond. It is commonly used to connect sensors, actuators, and other peripherals to microcontrollers and single-board computers like the Raspberry Pi. I2C is efficient for connecting multiple devices with limited pin resources and it is widely used in various applications, especially in the robotics field.

RASPBERRY PI SETUP

You can find a detailed guide on the commands to be executed on the donkey car's official Wiki. (Click here!) Don't worry it's just a copy-paste thing so there's not that much really to talk about :)

XMC 1100 BOOTKIT SETUP

Before getting into the code itself it is very important to make the disclaimer: Please update your XMC for Arduino to version 1.7.0 at least for this code to work!

To begin with, let's take a look at the variables we will be dealing with and what they will be used for:

• servo is an integer representing the servo pin.
• pwm is an integer used in servo control.
• Servo_pos and Servo_pos_aktuell are integers for the servo position.
• inString is a string for serial communication.
• neg is an integer for serial communication.
• speedvalue, speedvalue_aktuell, and speedflag are integers for motor control.
• on1, off1, on0, and off0 are integers for I2C communication.
• r and x are integers used in processing I2C communication.
• received is a boolean indicating whether I2C communication is received.

Of course, we will also have to include our half-bridge shield's library and the I2C Arduino library

#include <Wire.h>
#include "btn99x0_motor_control.hpp"
using namespace btn99x0;
// servo pin
int servo = 8;
// variables for servo control
int pwm;
int Servo_pos = 0;
int Servo_pos_aktuell;
// serial communication
String inString = "";
int neg;
//variables for motor control
int speedvalue = 0;
int speedvalue_aktuell;
int speedflag = false;
//i2c communication
int on1;
int off1;
int on0;
int off0;
int r;        //register
int x;        //value
int received;

The servoMove function moves a servo motor to a specified position. It takes two parameters, the first one servo specifies the pin that the servo motor is connected to, and the second one Servo_pos specifies the target position for the servo to move to.

The function starts by calculating the value of the pulse width modulation (PWM) signal that will be sent to the servo. It then sets the digital pin to high and waits for a duration equal to the PWM value. Finally, it sets the digital pin to low. This sequence of events results in a pulse that rotates the servo to the target position specified by Servo_pos.

void servoMove(int servo, int Servo_pos){
pwm = (Servo_pos) + 1500;
digitalWrite(servo, HIGH);
delayMicroseconds(pwm);
digitalWrite(servo, LOW);
}

We then create an instance of two classes of DCShield and MotorControl

DCShield shield; 
MotorControl btn_motor_control(shield);

Next, we jump into the setup loop!

We start initializing I2C and joining the bus with address #64. We setup up an event that is triggered whenever data is received from a master device and another whenever data is requested by a master device. Additionally, we will start serial communication with a baud rate of 9600. Finally, a delay of 10ms is added to ensure that communication is properly established.

Wire.begin(64);                // join i2c bus with address #8
Wire.onReceive(receiveEvent);  // register event
Wire.onRequest(requestEvent);  // register event
Serial.begin(9600);          // start serial for output
delay(10);

then within our setup loop, we'll configure our servo pins as output and call our servoMove function with the initialized Servo_pos variable to reset the servo's position.

pinMode(servo, OUTPUT);
servoMove(servo, Servo_pos);

The last thing to do before ending the setup loop is to initialize our motor

btn_motor_control.begin();
Serial.println("Initialising speed");
btn_motor_control.set_speed(speedvalue);
Serial.println("Starting motor");
Serial.println("Setup Done");

Now that the setup loop is out of the way let's go over the events that would be triggered within the I2C Communication

let's take a look at the receiveEvent event function (the one called every time data is received from the master device)

to avoid any trash data the function checks that the number of incoming bytes has to be equal to two in order to set the received flag to true.

void receiveEvent(int howMany) {
 if(howMany == 2){
 received = true;
 }
 else{
 received = false;
 }

The next thing to do is actually to read and process the data coming over I2C:

while (1 < Wire.available()) { // loop through first of two bytes
r = Wire.read();           // receive register byte
}
x = Wire.read();           // receive Value byte as an integer
//process
switch (r) {
case 8:
  off0 = x;
  speedflag = true;
  break;
case 9:
  if (speedflag == true) {
    off0 = off0+(x<<8);
    speedvalue = off0-255;
    speedflag = false;
  }
  break;
case 12:
  off1 = x;
  break;
case 13:
  off1 = off1+(x<<8);
  Servo_pos = off1-600;
  break;
 }
}

The first while loop saves the first byte into the variable r and then the second byte is saved onto x.

then the switch case dictates how the received data over I2C is processed and sets values for variables speedvalue, Servo_pos, and possibly off0and off1:

There are 4 possible cases, each with its own operation.

• If r is equal to 8, off0 is set to x and speedflag is set to true.
• If r is equal to 9 and speedflag is true, off0 is updated to off0 + (x << 8) (which shifts the bits of x 8 places to the left) and speedvalue is set to off0 - 255. Then speedflag is set to false.
• If r is equal to 12, off1 is set to x.
• If r is equal to 13, off1 is updated to off1 + (x << 8) and Servo_pos is set to off1 - 600.

The requestEvent() function in the code handles the data request from the master device (i.e. the I2C controller) in the I2C communication. It sends a response back to the master

void requestEvent() {
Wire.write(0x01); // respond with 0x01 as expected by master
Serial.println("0x01");
}

With all of this out of the way, the only thing remaining now is the main loop! This main loop of the program runs repeatedly. It checks if the received message from the I2C controller is valid by checking if "received" is equal to 0. If the received message is not valid, the message "fail!" is printed to the serial output.

void loop() {
if(received == 0) {
Serial.println("fail!");
}

Next, the code checks the difference between the current servo position and the desired servo position and if the difference is less than 200, the servo is moved to the desired position by calling the servoMove function. Then, the current servo position is updated to be the same as the desired position.

if(abs(Servo_pos_aktuell - Servo_pos)<200){
servoMove(servo, Servo_pos);
}
Servo_pos_aktuell = Servo_pos;

Similarly, the code checks the difference between the current speed value and the desired speed value and if the difference is less than 200, the motor speed is set to the desired value times 0.4 by calling the set_speed method of the btn_motor_control object. Finally, the current speed value is updated to be the same as the desired speed value.

if(abs(speedvalue_aktuell - speedvalue)<200){
btn_motor_control.set_speed(speedvalue*0.4);
}
speedvalue_aktuell = speedvalue;

A delay of 20 milliseconds is added at the end of the loop to avoid continuously running the loop and to ensure that the loop runs at a stable rate.

delay(20);
}

And by that, we would be done with the XMC 1100 bootkit setup!

MAKING SURE THAT EVERYTHING IS IN CHECK

After being done installing all the prerequisites of the Donkey Car software all you have to do now is to actually create your car module and start driving!

To do this simply in a terminal write the following command

donkey createcar --path *thenameyouwish*

then navigate into the new folder that was created

To make sure that your car does not need any calibration or modification you could always test your setup by actively controlling your car. You don't need need to connect your board to a fancy controller or anything.

after running your instance of your donkey car by going into your instantiated folder and typing

python manage.py drive

When running the donkey car software you will find on the terminal at the end a message with the address to hop into in our case here 8887

so all you gotta do now is hop onto your router's page and find your raspberry Pi's IP address (in this case it's 10.150.189.155)

After figuring that out go to any computer running on your wifi network (or even smartphone for that matter) go to the address bar and type your RaspberryPi's address: the address of the software.

In the cyan region at the top right corner, you could actually use your mouse to control the donkey car by clicking and dragging. This is very useful in making sure that the hardware is properly working!

Awesome! Now that your setup is running smoothly, it's time to start capturing some footage of your driving. This footage will be super helpful during the training process. There are lots of different ways to approach the driving process, and it's all up to you! For example, you could use your smartphone as a hotspot for your raspberry pi and control everything through your phone. And the best part? On the control screen, you'll see a start/stop recording button, so you can control what footage gets captured while you're driving. Another option is to connect a game controller wireless to your raspberry pi and use it to drive. The possibilities are endless! Click here for more info :)

Now to the Autonomous part!

Great job! You've just created something amazing. The next step is to teach your car how to drive like you do. This can be done by using the previously mentioned recorded footage of you driving and the Donkey Car UI to train a neural network with that footage.

The process is fairly simple but does require an external computer.(unfortunately the raspberry pi is not computationally strong enough to handle the training process)

All the footage you recorded will be stored in the data folder in your car's directory. To make things easier, we highly recommend using the Donkey Car UI. And don't worry, this step is extremely well-documented, so you'll have no trouble sorting it all out. Just click the link for more information!

For more instructions especially on the 3D printing and assembly side of the hardware feel free to check out our instructions document by clickingHERE.