How to Add a Coprocessor to Your JetBot

What Is a Coprocessor?

Imagine a robot that uses computer vision to locate a marker and drop off a package at a designated point. What kinds of tasks does it need to perform? First, it would need to gather camera data and put it through some kind of machine learning model to detect obstacles and targets. Next, the robot would need some way of controlling the motors via a driver, and also get feedback through rotation sensors. Finally, any sensor data would need to get read and processed, such as a capacitive touch interface or distance sensor. Performing all of these actions can be tedious and very time-consuming, as the robot, which is running an operating system, has to spend time communicating with peripherals rather than focusing on processing camera data.

A coprocessor can help alleviate some of this burden by directly communicating with peripherals and then sending only the important, processed information back. Additionally, it can receive commands and then handle tasks that are repetitive or take a long time to complete, such as moving actuators or toggling lighting systems.

The Jetson Nano and JetBot Platforms

The Jetson Nano has a Maxwell-based GPU that contains 128 CUDA cores that are capable at computing.5 teraflops per second (.5 TFLOPs). And with its powerful Arm Cortex-A57 processor and 4GB of LPDDR4 RAM, the Nano is a powerful computer in a small package. It also has support for numerous peripherals, including gigabit ethernet, HDMI 2.0, DisplayPort 1.4, 2 DSI connectors, an M.2 PCIe connector, 4 USB 3.0 ports, and two CSI camera connectors (for use with cameras like the Pi Camera). All of this connectivity makes it an ideal platform for AI at the edge projects.

One application for the Jetson Nano is the JetBot, which is a platform that NVIDIA created to showcase how robotics and machine learning could be combined into a small form-factor. The JetBot library is a series of packages and helpful Jupyter Notebooks that walk developers through the process of creating a robot. By default, the motors are controlled by a PCA9685 via I2C, but this creates extra overhead, and adding extra sensors and peripherals can slow down the JetBot even more, thus creating the need for a coprocessor.

Building a Protocol via UART

Sending data is great, but on its own, it doesn't mean anything; the programmer must give the data meaning. When designing a communication structure, think about the dynamic between the master and slave device(s) and what data needs to be sent and how often. For example, the coprocessor could send a packet of sensor values every half a second, then the Nano could decide what to do with it. It could also go the other way around, where the coprocessor receives commands from the Nano, such as motor values, and then sets the motors to that.

At a basic level, the master device uses a serial library to send certain values that a slave device receives. If the sensors you use will not change, then an identifier might not be necessary when sending every value at once. The slave device could just know that the first value corresponds to a temperature sensor, or that the third value comes from a gyroscope. But, if the values are not sent at the same time, then an identifying ID might be necessary. The master device could send two values for each sensor's data- an ID and accompanying value. The master and slave would both be programmed to agree on a common identifying value for each packet, letting them easily communicate at different times.

Modifying the JetBot Library

The JetBot Python library contains several files that handle the robot's functions, and they can be modified to send values over USB to a coprocessor, rather than an attached motor driver. For this example, the motor.py file will be changed to send motor values over serial.

First, all mentions of the Adafruit PCA9685 library are removed, since it won't be necessary. Next, a threading class is made called WriteVal, which when run, will call the underlying _write_value function. Threading is used to keep the process asynchronous, because a blocking function would keep other functions from running until all values had been pushed. The _write_value function simply calculates the motor's value and encodes it into the bytes datatype, along with an identifying value that corresponds to the channel's number. Finally, the _observe_value function is modified to create a new WriteValue object with all of the values to send, and runs it.

Microcontroller Program

Now that the microcontroller can receive information via USB/UART, it needs some processing to be useful. Simply make the slave device read information from serial, parse the data into two parts- an ID and value, and then feed it through a tree or a simple switch statement. For example, you could have an ID of 1 control the left motor, so you create a struct with two values and then assign their values to the return value of the parseInt function.

struct incomingDataContainer
{
    uint8_t id;
    int32_t value;
} incomingData;

incomingData.id = Serial.parseInt();
incomingData.value = Serial.parseInt();

What's Possible?

As previously mentioned, most menial tasks can be offloaded from the Nano to an external coprocessor. This can include controlling motors, reading from sensors, and handling interfaces.