Recognition-based Interactive Greeting System

Why I Build it？

I wanted to add some interactive and atmospheric elements to my large sign, so I built this project with FReeRTOS for 3 tasks.

What I have Built?

1. Recognition Task: I flashed a human detection model onto the Grove Vision AI Module V2.

2. Relay-Driven Motor Control Task: Driveing a motor via relay to open and close the petals

3. Rainbow LED Effect Task: Triggering a continuous rainbow breathing effect on the LED strip

When Grove Vision AI V2 Kit detects a person, it automatically triggers a continuous rainbow breathing effect on the LED strip and drives a motor via relay to open and close the petals. Progress has been a bit slow—I haven’t attached the petals yet—but you can look forward to the final look!

What I have Prepared?

I’ve already mentioned a few devices before, but I haven’t covered the LED strip yet—I went with a 12V one for extra brightness! And I bought the petal motor drive as a complete unit. It only has a power cable and no signal line, so an additional relay is needed to control it.

I’m only showing the modules and devices here, not the wiring, since my cables are way too messy!

How connect this hardware?

I will explain the wiring according to the three tasks separately.

1. Recognition Task

Use one Grove cable to connect Grove Vision AI Module V2 to the LED Driver Board.

2. Relay-Driven Motor Control Task

A Grove cable is still required, but the other end should be converted to a female Dupont connector to connect to the driver board. In my code, the D0 pin controls the relay, so the control signal wire from the Grove cable is connected to D0. I used a Grove expansion board with many small interfaces here because I want to control multiple petal motor units simultaneously. Therefore, a connection pathway also needs to be established between the Grove expansion board, the Grove relay, and the driver board.

Observant readers will notice that I used two ways to connect to the expansion board: one is direct wiring, and the other is via a Grove connector. Both methods work fine.

3. Relay-Driven Motor Control Task

5v to 5v; A0 to DIN; GND to GND

After connecting the three parts close to the LED driver board, you will notice that the GND line of the LED strip and the GND line of the expansion board both converge at the GND terminal of the driver board.

Now, as soon as you are detected in front of the camera, the lights will turn on and change colors, and the petals will bloom for you.

Complete Code

#include <Arduino.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/queue.h"
#include "freertos/semphr.h"
#include <Adafruit_NeoPixel.h>
#include <Wire.h>
#include <Seeed_Arduino_SSCMA.h>

// Grove Vision AI Module V2 I2C address
#define GROVE_VISION_ADDR 0x62

// Hardware pin definitions - for LED Driver Board for XIAO
#define LED_PIN          A0       // LED Driver Board fixed to D0 pin
#define LED_COUNT        60       // Number of LEDs, adjust according to actual LED strip
#define D2_PIN           D2       // D2 pin control
// I2C pins of XIAO ESP32S3
#define I2C_SDA_PIN      D4
#define I2C_SCL_PIN      D5

// Create objects
Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);
SSCMA AI;

// FreeRTOS task handles
TaskHandle_t visionTaskHandle;
TaskHandle_t ledTaskHandle;
TaskHandle_t d2ControlTaskHandle;

// Global variables and semaphores
volatile bool objectDetected = false;
SemaphoreHandle_t detectionMutex;
volatile unsigned long lastDetectionTime = 0; // Added: track the last detection time

// LED breathing rainbow effect parameters
int ledBrightness = 0;
bool brightnessDirection = true;
int rainbowHue = 0; // Hue value for rainbow effect

// Function declarations
void visionDetectionTask(void *pvParameters);
void ledControlTask(void *pvParameters);
void d2ControlTask(void *pvParameters);
uint32_t HSVtoRGB(int hue, int saturation, int brightness);
void breathingRainbowEffect();
void handleError(const char* errorMsg);

void setup() {
  Serial.begin(115200);

  // Initialize I2C communication (Grove Vision AI Module V2 uses I2C)
  // Explicitly set SDA and SCL pins to ensure proper hardware initialization
  Wire.begin(I2C_SDA_PIN, I2C_SCL_PIN);
  Wire.setClock(400000); // 400kHz I2C frequency
  delay(100); // Add delay to ensure bus stability

  // Initialize LEDs on the LED Driver Board
  strip.begin();
  strip.show(); // Initialize all LEDs to off
  strip.setBrightness(255); // Set max brightness, breathing effect controlled by software

  // Initialize D2 pin
  pinMode(D2_PIN, OUTPUT);
  digitalWrite(D2_PIN, LOW); // Initial state is LOW
  Serial.println("D2 pin initialized as OUTPUT");

  // LED hardware connection test - blink a few times on startup to confirm hardware works
  for (int i = 0; i < 3; i++) {
    strip.fill(strip.Color(255, 0, 0)); // Red
    strip.show();
    delay(200);
    strip.clear();
    strip.show();
    delay(200);
  }

  // Initialize Grove Vision AI Module with SSCMA library
  while (!AI.begin()) {
    Serial.println("Failed to initialize Grove Vision AI module. Retrying...");
    delay(1000);
  }
  Serial.println("Grove Vision AI Module V2 initialized successfully.");

  // Create mutex
  detectionMutex = xSemaphoreCreateMutex();

  // Create FreeRTOS tasks
  xTaskCreatePinnedToCore(
    visionDetectionTask,    // Task function
    "VisionTask",           // Task name
    4096,                   // Stack size
    NULL,                   // Task parameters
    2,                      // Priority
    &visionTaskHandle,      // Task handle
    0                       // CPU core (0 or 1)
  );

  xTaskCreatePinnedToCore(
    ledControlTask,
    "LEDTask",
    2048,
    NULL,
    1,
    &ledTaskHandle,
    1
  );

  // Create D2 pin control task
  xTaskCreatePinnedToCore(
    d2ControlTask,
    "D2Task",
    1024,                   // Smaller stack since task is simple
    NULL,
    1,                      // Same priority as LED task
    &d2ControlTaskHandle,
    0                       // CPU core 0
  );

  Serial.println("XIAO ESP32S3 Vision, LED and D2 Control System Started");
  Serial.println("Waiting for Grove Vision AI Module V2 detection...");
}

void loop() {
  // FreeRTOS task scheduling, main loop basically empty
  vTaskDelay(pdMS_TO_TICKS(1000));
}

// Vision detection task - using SSCMA library
void visionDetectionTask(void *pvParameters) {
  while (true) {
    bool hasObject = false;

    // Note: AI.invoke() returns 0 on success, non-zero on failure
    if (AI.invoke() == 0) {
      if (AI.boxes().size() > 0) {
        hasObject = true;
        Serial.print("Detected objects: ");
        Serial.println(AI.boxes().size());
      }
    } else {
        // If AI call fails, add delay to avoid frequent I2C communication errors
        Serial.println("AI.invoke() failed");
        vTaskDelay(pdMS_TO_TICKS(500)); 
    }
    
    if (xSemaphoreTake(detectionMutex, pdMS_TO_TICKS(100))) {
      bool previousState = objectDetected;
      
      // Keep detection state for 3 seconds to ensure LED task has enough time to respond
      if (hasObject) {
        lastDetectionTime = millis();
      }
      if (millis() - lastDetectionTime < 3000) {
        objectDetected = true;
      } else {
        objectDetected = false;
      }

      if (previousState != objectDetected) {
        if (objectDetected) {
          Serial.println("✅ Object detected! Starting LED and D2 tasks...");
        } else {
          Serial.println("❌ No object detected. Stopping LED and D2 tasks...");
        }
      }
      
      // Add state monitoring
      if (objectDetected && millis() % 2000 < 100) { // Print status every 2s
        Serial.println("🔥 LED and D2 should be active now!");
      }
      xSemaphoreGive(detectionMutex);
    }
    
    vTaskDelay(pdMS_TO_TICKS(100)); // Check every 100ms
  }
}

// LED control task
void ledControlTask(void *pvParameters) {
  const int breatheDelay = 30;   // Breathing effect delay (ms), smaller makes smoother

  while (true) {
    bool shouldRun = false;

    if (xSemaphoreTake(detectionMutex, pdMS_TO_TICKS(10))) {
      shouldRun = objectDetected;
      xSemaphoreGive(detectionMutex);
    }

    if (shouldRun) {
      breathingRainbowEffect();
      vTaskDelay(pdMS_TO_TICKS(breatheDelay));
    } else {
      strip.clear();
      strip.show();
      vTaskDelay(pdMS_TO_TICKS(100));
    }
  }
}

// D2 pin control task
void d2ControlTask(void *pvParameters) {
  while (true) {
    bool shouldActivate = false;

    // Get detection state
    if (xSemaphoreTake(detectionMutex, pdMS_TO_TICKS(10))) {
      shouldActivate = objectDetected;
      xSemaphoreGive(detectionMutex);
    }

    if (shouldActivate) {
      // When object detected, set D2 pin HIGH and hold
      digitalWrite(D2_PIN, HIGH);
      vTaskDelay(pdMS_TO_TICKS(50)); // Check every 50ms
    } else {
      // When no object detected, set D2 pin LOW
      digitalWrite(D2_PIN, LOW);
      vTaskDelay(pdMS_TO_TICKS(100)); // Check every 100ms
    }
  }
}

// HSV to RGB conversion function
uint32_t HSVtoRGB(int hue, int saturation, int brightness) {
  // hue: 0-359, saturation: 0-255, brightness: 0-255
  int r, g, b;
  
  if (saturation == 0) {
    r = g = b = brightness;
  } else {
    int sector = hue / 60;
    int remainder = (hue % 60) * 255 / 60;
    int p = (brightness * (255 - saturation)) / 255;
    int q = (brightness * (255 - (saturation * remainder) / 255)) / 255;
    int t = (brightness * (255 - (saturation * (255 - remainder)) / 255)) / 255;
    
    switch(sector) {
      case 0: r = brightness; g = t; b = p; break;
      case 1: r = q; g = brightness; b = p; break;
      case 2: r = p; g = brightness; b = t; break;
      case 3: r = p; g = q; b = brightness; break;
      case 4: r = t; g = p; b = brightness; break;
      default: r = brightness; g = p; b = q; break;
    }
  }
  
  return strip.Color(r, g, b);
}

// Breathing rainbow effect function
void breathingRainbowEffect() {
  // Breathing effect: brightness changes
  if (brightnessDirection) {
    ledBrightness += 3; // Smaller step, smoother breathing
    if (ledBrightness >= 255) {
      ledBrightness = 255;
      brightnessDirection = false;
    }
  } else {
    ledBrightness -= 3;
    if (ledBrightness <= 20) { // Keep minimum brightness > 0 for faint glow
      ledBrightness = 20;
      brightnessDirection = true;
    }
  }

  // Rainbow effect: hue changes
  rainbowHue += 1; // Hue change speed
  if (rainbowHue >= 360) {
    rainbowHue = 0;
  }

  // Apply same color and brightness to entire strip
  uint32_t color = HSVtoRGB(rainbowHue, 255, ledBrightness);
  strip.fill(color);
  strip.show();
}

// Error handling function (optional)
void handleError(const char* errorMsg) {
  Serial.print("Error: ");
  Serial.println(errorMsg);

  for (int i = 0; i < 3; i++) {
    strip.fill(strip.Color(255, 0, 0));
    strip.show();
    vTaskDelay(pdMS_TO_TICKS(200));
    strip.clear();
    strip.show();
    vTaskDelay(pdMS_TO_TICKS(200));
  }
}