Published October 18, 2022 © MIT

Smart Air Cooler using Person Sensor.Ft Useful Sensor

Baking Intelligence into a trivial Air Cooler using Person Sensor by Useful Sensor

IntermediateFull instructions provided2,070

Smart Air Cooler using Person Sensor.Ft Useful Sensor

Things used in this project

Hardware components

Useful Sensors Person Sensor

Seeed Studio Seeed XIAO BLE nRF52840 Sense

Seeed Studio Seeeduino XIAO Expansion board

SG90 Micro-servo motor

Seeed Studio Grove - Relay

Seeed Studio Grove - 4 pin Male Jumper to Grove 4 pin Conversion Cable

Software apps and online services

PlatformIO IDE

Hand tools and fabrication machines

Crompton Air Buddy

Multitool, Screwdriver

Set Screw, Pack of 10

Hot glue gun (generic)

Story

Introduction

Have you ever thought, how easy and handy would it become to enable your Devices with Edge AI without having to go through the cumbersome process of integrating a library or having software engineering resources for ML Stacks? Yes, we do have tools to make this process less arduous. However, what if a sensor could directly present extracted features of image data? Wouldn't that be interesting?

Person Sensor from Useful Sensor is an edgeAI device/sensor that has capable of detecting a person in the camera frame and generates details on who (you can set IDs for multiple people) and where the person is in the frame. It does all the heavy lifting of capturing, identifying the person, computing the NMS algorithm, and handing over the indispensable data over I2C when requested through simple enough APIs

As every other device is getting smarter in this Era why not add intelligence to our dear old Summer friend Air Coolers? Air Buddy from Crompton is specifically designed for kitchen usage. It was meant to provide directed airflow toward the Cook. This would not have any effect on the flame of the stove thus not affecting the cooking process. Despite having such good airflow and swing features, it did lack to provide proper cooling if he/she moves throughout the kitchen.

So, I thought of adding Vision to our Blind Air Buddy. Now, it can detect the presence of the person - "turn on the fan". Track the movement of the person throughout the frame and adjust the blades accordingly to provide the user with efficient cooling. All these upgrades by just one sensor.

Working

The goal here is to upgrade the Air Cooler and adjust its blades so as to direct the airflow toward the user in the frame. To add vision to the existing project the general procedure is as follows:

Capture images from a vision sensor.
Process it on MCU.
Implement edge-AI algorithms to first detect if a person is present in the frame and secondly compute the relative coordinates of the person in the frame.
Use this generated data to position the servo motors.

However, it's easier than being said, specifically for the first three steps. The person sensor does the heavy lifting for the first three steps and provides the user with APIs to easily integrate the module into the solution. Internally the Person Sensor bundles an image sensor and a small microcontroller into a single board in which the user can treat any other sensor, giving it power and getting information back through a standard I2C hardware interface. The sensor also supports generating an interrupt on the TP1 pin when a person is detected. This can act as an external trigger to wake an MCU from sleep and start processing the data.

The air cooler was modified with the addition of two servo motors for vertical and horizontal movement. Due to the mechanical structure of the cooler, we need to limit the range of motion of the servo motors so as to avoid any mechanical stress and breakdown of the motors.

Using the visual information received from the Person Sensor, the centroid of the bounding box is calculated. These centroid coordinates are then mapped to the actual position of servo motors. Also depending on the confidence rating, we control the relay which turns the Air Cooler on or off.

Hardware Build:

Seeed Studio Xiao Ble is used to coalesce all the data from the sensor and control the servo motors accordingly. Using the expansion boards exposes all the needed pin in the grove form factor and also keep the system neat and tidy. Since the person sensor supports I2C protocol and has qwiic connector, using a grove to qwiic connector, it was connected to one of the I2C grove connectors on the expansion board. The power supply for the Xiao is capable enough to power two servo motors. Hence, both motors were connected to Grove connectors with pins A0 and A7 pins. You can change them according to your liking. The relay, responsible for turning on and off the air cooler, is connected to the D10 pin.

Thin malleable steel rods were used for connecting the vertical flaps to the servo. For the horizontal movement, a circular disk with a crank pin was attached to the servo motor. The motors were fixed in position using some nuts, bolts, and a glue gun. After fixing the mechanical design, the whole system was connected using grove jumper wires to keep the system clean and tidy. A section of the power supply cable of the air cooler was cut into half to insert the relay module so as to control the power supply.

For demonstration purposes, the system is powered from a USB power bank connected to Xiao MCU. However, the power could have also been drawn from the Air Cooler as well. Here are some snapshots of the hardware built:

1 / 8

Software Build:

Since most of the computation is taken care of by the person sensor, hence the main code is quite straightforward. The software collects metadata of the position of the person in the frame through I2C and computes the centroid from the same information. The relative coordinates are then mapped to the actual motor position to direct airflow. The confidence percentage and the person detection status are also displayed on the monochrome screen of the expansion board which shares the I2C bus with the Person Sensor.

The mbed I2C requestfrom() function doesn't behave properly as expected. Hence, we would need to add the following code segment in the Wire.cpp file in the Arduino Library:

size_t arduino::MbedI2C::requestFrom(uint8_t address, size_t len, bool stopBit) {
char buf[256];
rxBuffer.clear();  //Add This.

The software was built using PlatformIO, however, should work fine on the Arduino IDE as well. You can follow this guide to set up the Xiao BLE environment in the Arduino IDE. Below are the links to the project repository and dependent libraries:

Project Code: https://github.com/Pratyush-Mallick/person_sensor_xiao
Person Sensor Drivers: https://github.com/usefulsensors/ID-Demo/blob/main/personID/I2CDriver.h
Display Drivers: https://github.com/olikraus/u8g2

Demonstration

The moment of truth is here. Below is a playlist of the videos showcasing the Smart Air Cooler powered by the Person Sensor:

System Walkthrough:

Internal Working:

Cooler Action Part I & II:

Future Scope:

It's a promising device and has a lot to offer that I haven't explored yet. For example, the sensor can be trained on the device to detect multiple persons assigned to different IDs and would provide the bounding box information for each detected person.

The sensor opens up a plethora of opportunities for baking intelligence into existing devices.

For more information on the sensor, you can reach out to Pete Warden or drop an email to contact@usefulsensors.com

Code

Main Code Segment

#include <Arduino.h>
#include <U8x8lib.h>
#include <Wire.h>
#include <Servo.h>
#include "I2CDriver.h"
 
U8X8_SSD1306_128X64_NONAME_HW_I2C u8x8(/* clock=*/ PIN_WIRE_SCL, /* data=*/ PIN_WIRE_SDA, /* reset=*/ U8X8_PIN_NONE);   // OLEDs without Reset of the Display

I2CDriver i2c = I2CDriver();
static inference_results_t results;
static uint8_t centroid_x;
static uint8_t centroid_y;

Servo servo_x; // A7 <--- 140 left X 0 right -->  Horizontal Movement mapped to X cetroid.
Servo servo_y;  // A0 <--- 60 down Y 0 up  -->  Vertical Movement mapped to Y centroid.

// Maximum servo movement restricted mechanical design.
#define MIN_SERVO      0
#define MAX_SERVO_X    140  
#define MAX_SERVO_Y    60

// Since the camera has a wider field of view, 
// which the servo motors can't match.
// Hence we need to capt the position coordinates.
#define MIN_CENTROID   50
#define MAX_CENTROID   200
#define FAN_SWITCH     D10

uint8_t motor_x_position;
uint8_t motor_y_position;

void setup(void) {

  MbedI2C(D4, D5);
  u8x8.begin();
  u8x8.setFlipMode(1);

  // Configuring the Person Sensor.
  i2c.setMode(i2c.MODE_CONTINUOUS);
  i2c.setIdModelEnabled(true);

  servo_x.attach(D7);
  servo_y.attach(A0); 
  pinMode(FAN_SWITCH, OUTPUT);

  u8x8.setFont(u8x8_font_chroma48medium8_r);
  u8x8.setCursor(0, 0);
  u8x8.print("Confidence: ");
}
 
void loop(void) {
  results = i2c.read();
  u8x8.setCursor(11, 0);
  u8x8.print(results.boxes->confidence);
  u8x8.setCursor(0, 2);
  u8x8.print("Person Detected:");
  u8x8.setCursor(0, 4);

  if (results.boxes->confidence > 90) {
    // Turn on the FAN.
    digitalWrite(FAN_SWITCH, HIGH);

    centroid_x = (results.boxes->data[2] + results.boxes->data[0])/2;
    centroid_y = (results.boxes->data[3] + results.boxes->data[1])/2;

    // Limiting the centroid range to avoid overstressing the motor
    // due to mechanical design.
    centroid_x = centroid_x > MAX_CENTROID ? MAX_CENTROID : centroid_x;
    centroid_y = centroid_y > MAX_CENTROID ? MAX_CENTROID : centroid_y;

    // Mapping the positonal coordinates to servo positions.
    motor_x_position = map(centroid_x, MIN_CENTROID, MAX_CENTROID, MIN_SERVO,        MAX_SERVO_X);
    motor_y_position = map(centroid_y, MIN_CENTROID, MAX_CENTROID, MIN_SERVO,        MAX_SERVO_Y);
    servo_x.write(motor_x_position);
    servo_y.write(motor_y_position);

    u8x8.print("Yes"); 
  } else {
    u8x8.print("No!");
    digitalWrite(FAN_SWITCH, LOW);  
    servo_x.write(MAX_SERVO_X/2);
    servo_y.write(MAX_SERVO_Y/2);
  }
}

Credits

Pratyush Mallick

4 projects • 32 followers

I love tinkering, taking things apart, and making them work. Happy Hacking