Published November 28, 2025 © MIT

RT-Thread - Vision Board Basic Color Sorting w/ Arduino

Smart Color-Tracking Actuator that bridges the gap between high-speed computer vision and physical robotics. Using an OpenMV Cam & Arduino

BeginnerShowcase (no instructions)3 hours13

RT-Thread - Vision Board Basic Color Sorting w/ Arduino

Things used in this project

Hardware components

RT-Thread Vision Board

Arduino Nano

SG90 Micro-servo motor

Software apps and online services

Arduino IDE

OpenMV IDE

Story

The Crisis (The "Why")

I was supposed to showcase an RT-Thread Etherkit board for a school event, but we hit a wall immediately the board required a dedicated programmer hardware that we simply didn't have. My prof showed some board to use instead during our lecture, one showed my pique interest RT-Thread Vision Board. It had a camera. It looked interesting.

I pivoted immediately. Instead of a complex IoT demo, I decided to build something visual and tangible: A Smart Color Sorting Robot. In an industrial setting, this is the logic that sorts packages or separates plastic types for recycling. For me, it was a way to save the presentation.

The "AI" Trap (The "How")

Like everyone else in 2025, my first thought was "Let's use AI!"

I jumped into Edge Impulse to train a Machine Learning model. Ideally, it would learn to recognize "Red Object" vs "Blue Object." In reality? It was a disaster.

Model 1: Detected everything as "Brown" or "Black."
Model 2: I improved the dataset and got 70% accuracy on paper. But in the real world, it was confident but wrong. It would look at a red ball and scream "Blue!"

I realized I was over-engineering it. I didn't need a neural network to tell me what color a pixel was. I scrapped the AI and switched to Hue Detection (Math-based color tracking). It was faster, didn't need training, and was incredibly sensitive to color changes. Sometimes, the old ways are better.

The Language Barrier & The Arduino Bridge

The Vision Board is powerful, but there was a catch: the documentation is almost entirely in Chinese, and English support is non-existent.

I needed to move a Servo motor. On the Vision Board, exposing the PWM pins required navigating a new IDE (RT-Thread Studio) and digging through translated datasheets. I didn't have time for that.

The Solution: I used an Arduino Nano as a "Translator." The Vision Board does the heavy lifting (Vision Processing) and sends simple text commands over UART to the Arduino. The Arduino which has thousands of English tutorials that handles the Servo.

The "Vibe Code" Moments (The Struggle)

Connecting the two boards was where the real headache began.

1. The MOSFET Mistake: The Vision Board sends 3.3V signals. The Arduino reads 5V. I tried to use an IRF540N MOSFET as a logic level shifter. Bad idea. That MOSFET is designed for driving motors, not high-speed data. It completely blocked the signal.

2. The Baud Rate Jitter: I set the communication speed to 115200 thinking "faster is better." Instead, the servo twitched and jittered like it was possessed. It turns out SoftwareSerial on Arduino can't handle those speeds. Dropping to 9600 fixed it instantly.

3. The "Ghost" Signals: Initially, the code sent a message for every frame. "Red, Red, Red, Red..." 30 times a second. The Arduino buffer got clogged, and the servo lagged behind reality. I had to write a "State Change" logic so the camera only speaks when the color changes.

The Victory Lap (Sort of)

It’s 3:00 AM. The code finally compiles. I hold up a red object.

The Vision Board sees it.

The Arduino receives the signal.

The Servo snaps to 40 degrees.

Honestly? My reaction wasn't "Eureka!" It was "Meh... finally."

I don't have a 3D-printed chassis, and the wires are a mess, but the logic holds up. If I had another month, I’d add a conveyor belt and an LCD screen to make it a true industrial prototype. But for a last-minute pivots to save a school symposium? It works.Authors Note: Generative AI are being used.

Code

import sensor, image, time, math
from pyb import UART

# --- UART SETUP ---
uart = UART(2, 9600, timeout_char=200)

# --- SENSOR SETUP ---
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_windowing((240, 240))

# --- 180 DEGREE ROTATION ---
sensor.set_vflip(True)
sensor.set_hmirror(True)
# ---------------------------

sensor.skip_frames(time=2000)

# --- COLOR THRESHOLDS ---
thresholds = [
    (0, 100, 56, 95, 41, 74),       # 0: Red-ish
    (0, 100, -128, -22, -128, 99),  # 1: Green-ish
    (0, 100, -128, 98, -128, -16)   # 2: Blue-ish
]

# --- HUE CALCULATION ---
def get_hue(r, g, b):
    r, g, b = r/255.0, g/255.0, b/255.0
    mx = max(r, g, b)
    mn = min(r, g, b)
    df = mx - mn
    if mx == mn:
        h = 0
    elif mx == r:
        h = (60 * ((g - b) / df) + 360) % 360
    elif mx == g:
        h = (60 * ((b - r) / df) + 120) % 360
    elif mx == b:
        h = (60 * ((r - g) / df) + 240) % 360
    return int(h)

clock = time.clock()

# --- STATE VARIABLE ---
# Stores the last color sent to the Arduino
last_sent_color = "None"

while(True):
    clock.tick()
    img = sensor.snapshot()

    # Find blobs
    blobs = img.find_blobs(thresholds, pixels_threshold=200, area_threshold=200, merge=True)

    current_color = "None" # Default if nothing is found
    largest_blob = None

    if blobs:
        # 1. Find the Largest Blob (Ignores small noise)
        largest_blob = max(blobs, key=lambda b: b.pixels())

        # 2. Get Color Stats
        stats = img.get_statistics(roi=largest_blob.rect())
        rgb_tuple = image.lab_to_rgb((stats.l_mean(), stats.a_mean(), stats.b_mean()))
        hue = get_hue(rgb_tuple[0], rgb_tuple[1], rgb_tuple[2])

        # 3. Determine Color Name
        if 330 <= hue or hue < 25:
            current_color = "Red"
        elif 25 <= hue < 85:
            current_color = "Yellow"
        elif 85 <= hue < 160:
            current_color = "Green"
        elif 160 <= hue < 260:
            current_color = "Blue"
        elif 260 <= hue < 330:
            current_color = "Violet"
        else:
            current_color = "Unknown"

    # --- INSTANT SEND LOGIC ---
    # If the color we see NOW is different from the LAST one we sent...
    if current_color != last_sent_color:

        # Don't send "None" or "Unknown" to Arduino, just reset the state
        if current_color != "None" and current_color != "Unknown":
            uart.write(current_color + "\n")
            print("UART Sent:", current_color)

        # Update the state immediately
        last_sent_color = current_color


    # --- DRAWING ---
    if largest_blob and current_color != "None":
        img.draw_rectangle(largest_blob.rect(), color=(255, 255, 255), thickness=3)
        img.draw_string(largest_blob.x(), largest_blob.y() - 15, current_color, color=(255, 255, 255))

    print(clock.fps(), "fps")

#include <SoftwareSerial.h>
#include <Servo.h>

// --- CONFIGURATION ---
// CHANGED: Use 9600 Baud for stability with SoftwareSerial
SoftwareSerial camSerial(2, 3); 

Servo myServo;
const int SERVO_PIN = 9; 

const int CENTER_POS = 75; 
const int RED_POS = 40;
const int BLUE_POS = 110;

String lastColor = "";

void setup() {
  // CHANGED: Lowered to 9600
  Serial.begin(9600);
  camSerial.begin(9600); 

  myServo.attach(SERVO_PIN);
  
  Serial.println("--- Initialization ---");
  myServo.write(RED_POS);
  delay(500); // Reduced delay for faster startup
  myServo.write(BLUE_POS);
  delay(500); 
  myServo.write(CENTER_POS); 
  delay(500);

  Serial.println("Ready.");
}

void loop() {
  if (camSerial.available() > 0) {
    
    // --- FIX: READ THE FRESHEST DATA ---
    // If the camera sent multiple messages while we were moving the servo,
    // the buffer might contain: "Red\nRed\nBlue\n"
    // We want the LAST one, not the first one.
    
    String colorName = camSerial.readStringUntil('\n');
    colorName.trim(); 
    
    // Quick loop to clear any remaining queued messages
    // This ensures we don't react to old history
    while(camSerial.available() > 0) {
       char t = camSerial.read(); 
    }

    if (colorName.length() > 0) {
      
      if (colorName != lastColor) {
        
        Serial.print("New: ");
        Serial.println(colorName);

        // 1. Reset to Center
        myServo.write(CENTER_POS);
        delay(300); // Reduced slightly for snappiness

        // 2. Determine Target
        int targetAngle = -1; 
        
        if (colorName == "Red") targetAngle = RED_POS;
        else if (colorName == "Blue") targetAngle = BLUE_POS;

        // 3. Action
        if (targetAngle != -1) {
           Serial.print("Moving -> ");
           Serial.println(targetAngle);
           myServo.write(targetAngle);
           
           // Waiting time (Action Duration)
           delay(1000);
           
           // Return to Center
           myServo.write(CENTER_POS);
        }

        lastColor = colorName;
      }
    }
  }
}

Credits

PERCH ARNEL II MONTEFALCON

1 project • 0 followers

Thanks to Supachai Vorapojpisut.

RT-Thread - Vision Board Basic Color Sorting w/ Arduino

Things used in this project

Hardware components

Software apps and online services

Story

The Crisis (The "Why")

The "AI" Trap (The "How")

The Language Barrier & The Arduino Bridge

The "Vibe Code" Moments (The Struggle)

The Victory Lap (Sort of)

Schematics

Circuit Diagram

Code

OpenMV Code for Vision Board

Arduino Code for communicating OpenMV board and SG90 servo

Credits

PERCH ARNEL II MONTEFALCON

Comments

Embed the widget on your own site

RT-Thread - Vision Board Basic Color Sorting w/ Arduino

RT-Thread - Vision Board Basic Color Sorting w/ Arduino

Things used in this project

Hardware components

Software apps and online services

Story

The Crisis (The "Why")

The "AI" Trap (The "How")

The Language Barrier & The Arduino Bridge

The "Vibe Code" Moments (The Struggle)

The Victory Lap (Sort of)

Schematics

Circuit Diagram

Code

OpenMV Code for Vision Board

Arduino Code for communicating OpenMV board and SG90 servo

Credits

PERCH ARNEL II MONTEFALCON

Comments

Related channels and tags