Kutluhan Aktar
Published © CC BY

IoT AI-driven Poultry Feeder and Egg Tracker w/ WhatsApp

Detect when the poultry feeder needs to be refilled and track unhatched eggs in the coop w/ object detection to get informed via WhatsApp.

ExpertFull instructions provided4,879

Things used in this project

Hardware components

WizFi360-EVB-Pico
WIZnet WizFi360-EVB-Pico
×1
OpenMV Cam H7
×1
LattePanda 3 Delta
LattePanda 3 Delta
×1
DFRobot 8.9' 1920x1200 IPS Touch Display
×1
ST7735 1.8' Color TFT Display
×1
DHT22 temperature-humidity sensor
Adafruit DHT22 temperature-humidity sensor
×1
Creality CR-200B 3D Printer
×1
MicroSD Card (FAT32)
×1
SparkFun Button (6x6)
×2
Solderless Breadboard Half Size
Solderless Breadboard Half Size
×2
DC Barrel Jack Adapter - Female
×1
Xiaomi 20000 mAh 3 Pro Type-C Power Bank
×1
Jumper wires (generic)
Jumper wires (generic)
×1

Software apps and online services

Edge Impulse Studio
Edge Impulse Studio
OpenMV IDE
Arduino IDE
Arduino IDE
Fusion 360
Autodesk Fusion 360
Ultimaker Cura

Hand tools and fabrication machines

Hot glue gun (generic)
Hot glue gun (generic)
Soldering iron (generic)
Soldering iron (generic)
Solder Wire, Lead Free
Solder Wire, Lead Free

Story

Read more

Custom parts and enclosures

poultry_feeder_and_unhatched_egg_tracker_main_case.stl

poultry_feeder_and_unhatched_egg_tracker_cover.stl

poultry_feeder_and_unhatched_egg_tracker_camera.stl

poultry_sandbox.stl

Edge Impulse Model (OpenMV Firmware)

poultry_feeder_and_egg_tracker.zip

Schematics

OpenMV Cam H7

WizFi360-EVB-Pico

Code

poultry_egg_tracker_data_collect.py

Python
# IoT AI-driven Poultry Feeder and Unhatched Egg Tracker w/ WhatsApp
#
# OpenMV Cam H7
#
# Detect when the poultry feeder needs to be refilled and track unhatched eggs
# in the coop w/ object detection to get informed via WhatsApp.
#
# By Kutluhan Aktar

import sensor, image, lcd
from pyb import RTC, Pin, LED
from time import sleep


# Initialize the camera sensor.
sensor.reset()
sensor.set_pixformat(sensor.RGB565) # or sensor.GRAYSCALE
sensor.set_framesize(sensor.QQVGA2) # Special 128x160 framesize for LCD Shield.

# Initialize the ST7735 1.8" color TFT screen.
lcd.init()

# Set the built-in RTC (real-time clock).
rtc = RTC()
rtc.datetime((2022, 8, 30, 2, 12, 25, 0, 0))

# RGB:
red = LED(1)
green = LED(2)
blue = LED(3)

# Buttons:
button_a = Pin("P6", Pin.IN, Pin.PULL_UP)
button_b = Pin("P1", Pin.IN, Pin.PULL_UP)

# Capture an image with the OpenMV Cam H7 as a sample.
def save_sample(name, color, leds):
    # Get the current date and time.
    date = rtc.datetime()
    date = ".{}_{}_{}_{}-{}-{}".format(date[0], date[1], date[2], date[4], date[5], date[6])
    # Take a picture with the given frame settings (QVGA).
    sensor.set_framesize(sensor.QVGA)
    sample = sensor.snapshot()
    sleep(1)
    # Save the captured image.
    file_name = "/samples/" + name + date + ".jpg"
    sample.save(file_name, quality=20)
    if leds[0]: red.on()
    if leds[1]: green.on()
    if leds[2]: blue.on()
    print("\nSample Saved: " + file_name + "\n")
    # Show a glimpse of the captured image on the ST7735 1.8" color TFT screen.
    sensor.set_framesize(sensor.QQVGA2)
    lcd_img = sensor.snapshot()
    lcd_img.draw_rectangle(0, 0, 128, 30,fill=1, color =(0,0,0))
    lcd_img.draw_string(int((128-16*len(name))/2), 3, name, color=color, scale=2)
    lcd_img.draw_rectangle(0, 130, 128, 160, fill=1, color =(0,0,0))
    lcd_img.draw_string(int((128-16*len("Saved!"))/2), 132, "Saved!", color=color, scale=2)
    lcd_img.draw_cross(64, 80, color=color, size=8, thickness=2)
    lcd.display(lcd_img)
    sleep(5)
    red.off()
    green.off()
    blue.off()

while(True):
    # Display a real-time video stream on the ST7735 1.8" color TFT screen.
    sensor.set_framesize(sensor.QQVGA2)
    lcd_img = sensor.snapshot()
    lcd.display(lcd_img)
    # Save samples.
    if(button_a.value() == False):
        save_sample("Egg", (255,0,255), (1,0,1))
    if(button_b.value() == False):
        save_sample("Feeder", (0,255,0), (0,1,0))

poultry_egg_tracker_run_model.py

Python
# IoT AI-driven Poultry Feeder and Unhatched Egg Tracker w/ WhatsApp
#
# OpenMV Cam H7
#
# Detect when the poultry feeder needs to be refilled and track unhatched eggs
# in the coop w/ object detection to get informed via WhatsApp.
#
# By Kutluhan Aktar

import sensor, image, os, tf, math, uos, gc, lcd
from time import sleep
from pyb import RTC, LED, UART


sensor.reset()
sensor.set_pixformat(sensor.RGB565)    # or sensor.GRAYSCALE
sensor.set_framesize(sensor.QQVGA2)    # Special 128x160 framesize for LCD Shield.
sensor.skip_frames(time=2000)          # Let the camera adjust.

# Define the required parameters to run an inference with the Edge Impulse FOMO model.
net = None
labels = None
min_confidence = 0.7

# Include the Edge Impulse FOMO model converted to an OpenMV firmware.
# Then, print errors, if any.
try:
    labels, net = tf.load_builtin_model('trained')
except Exception as e:
    raise Exception(e)

# Define the unique color codes for each class (egg and feeder).
colors = [
    (255, 255, 255),
    (255, 0, 255),
    (0, 255, 0),
]

# Initiate the integrated serial port on the OpenMV Cam H7.
uart = UART(3, 115200, timeout_char=1000)

# Initialize the ST7735 1.8" color TFT screen.
lcd.init()

# Set the built-in RTC (real-time clock).
rtc = RTC()
rtc.datetime((2022, 8, 30, 2, 12, 29, 30, 0))

# RGB:
red = LED(1)
green = LED(2)
blue = LED(3)

# Define the data holders.
eggs = 0
feeder = 0
feeder_status = "EMPTY"

while(True):
    # Get the current date and time.
    date = rtc.datetime()
    m, s = (int(date[5]), int(date[6]))
    date = "{}_{}_{}.{}-{}-{}".format(date[0], date[1], date[2], date[4], date[5], date[6])
    # Take a picture with the given frame settings (QQVGA2).
    img = sensor.snapshot()

    # Run inference to detect unhatched eggs and whether the poultry feeder needs to be refilled.
    # Via the detect function, obtain all detected objects found in the recently captured image, split out per label (class).
    for i, detection_list in enumerate(net.detect(img, thresholds=[(math.ceil(min_confidence * 255), 255)])):
        # Exclude the class index 0 since it is the background class.
        if (i == 0): continue

        # If the Edge Impulse FOMO model predicted a label (class) successfully:
        if (len(detection_list) == 0): continue

        # Clear the egg and feeder detection counters.
        if(i==1): eggs = 0
        feeder = 0

        # Get the prediction (detection) results for each label (class).
        print("\n********** %s **********" % labels[i])
        for d in detection_list:
            # Update the egg and feeder detection counters.
            if(i==1): eggs+=1
            if(i==2): feeder+=1
            # Draw a circle in the center of the detected objects with the assigned label (class) colors.
            [x, y, w, h] = d.rect()
            center_x = math.floor(x + (w / 2))
            center_y = math.floor(y + (h / 2))
            img.draw_circle((center_x, center_y, 12), color=colors[i], thickness=2)
            print('c: (%d, %d)' % (center_x, center_y))

    # Evaluate the feeder status by utilizing the feeder detection counter.
    if(feeder>0): feeder_status = "OK"
    if(feeder==0): feeder_status = "EMPTY"

    # Each half an hour, transfer the detection results to the WizFi360 module via the serial port to notify the user with a WhatsApp message.
    if((m,s) == (0,0) or (m,s) == (30,0)):
        query = "&egg_count=%d&feeder_status=%s" % (eggs, feeder_status)
        uart.write(query)
        print("\n\nResults transferred to the WizFi360 module.")
        print(query)
        blue.on()
        sleep(5)
        blue.off()


    # Display the egg detection counter, the feeder status, and each detected object on the ST7735 1.8" color TFT screen.
    s_eggs = "Egg Count: " + str(eggs)
    s_feeder = "Feeder: " + feeder_status
    img.draw_rectangle(0, 0, 128, 15,fill=1, color =(0,0,0))
    img.draw_string(int((128-8*len(s_eggs))/2), 3, s_eggs, color=(255,0,255), scale=1)
    img.draw_rectangle(0, 145, 128, 160, fill=1, color =(0,0,0))
    img.draw_string(int((128-8*len(s_feeder))/2), 147, s_feeder, color=(0,255,0), scale=1)
    lcd.display(img)

Poultry_Feeder_and_Unhatched_Egg_Tracker.ino

Arduino
         /////////////////////////////////////////////  
        //      IoT AI-driven Poultry Feeder       //
       //  and Unhatched Egg Tracker w/ WhatsApp  //
      //             ---------------             //
     //           (WizFi360-EVB-Pico)           //           
    //             by Kutluhan Aktar           // 
   //                                         //
  /////////////////////////////////////////////

//
// Detect when the poultry feeder needs to be refilled and track unhatched eggs in the coop w/ object detection to get informed via WhatsApp.
//
// For more information:
// https://www.theamplituhedron.com/projects/IoT_AI_driven_Poultry_Feeder_and_Unhatched_Egg_Tracker_w_WhatsApp
//
//
// Connections
// WizFi360-EVB-Pico :  
//                                DHT22 Temperature and Humidity Sensor
// D28 --------------------------- DATA
// 3.3V -------------------------- VCC
// GND --------------------------- GND
//                                OpenMV Cam H7
// D13 --------------------------- P4
// D12 --------------------------- P5


// Include the required libraries.
#include "WizFi360.h"
#include "SoftwareSerial.h"
#include "DHT.h"

char ssid[] = "[SSID]";       // your network SSID (name)
char pass[] = "[PASSWORD]";   // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;             // your network key Index number (needed only for WEP)

// Change the server below before running the code.
IPAddress server(192, 168, 1, 22);

// Define the web application path.
String application = "/poultry_feeder_and_egg_tracker/get_data.php";

// Define a software serial port to communicate with the integrated WizFi360 module. 
SoftwareSerial WizFi360(5, 4); // RX, TX
// Define a software serial port to communicate with the OpenMV Cam H7.
SoftwareSerial OpenMV(13, 12); // RX, TX

//  Initialize the Ethernet client object.
WiFiClient client;

// Define the DHT22 temperature and humidity sensor settings and the DHT object.
#define DHTPIN 28
#define DHTTYPE DHT22
DHT dht(DHTPIN, DHTTYPE);

// Define the data holders. 
#define SERIAL_BAUDRATE  115200
int status = WL_IDLE_STATUS;
float humidity, temperature, hic;
String OpenMV_data = "";

void setup(){
  // Initialize software serial ports.
  WizFi360.begin(SERIAL_BAUDRATE);
  OpenMV.begin(SERIAL_BAUDRATE);
  Serial.begin(SERIAL_BAUDRATE);
  
  // Initialize the integrated WizFi360 module. 
  WiFi.init(&WizFi360);

  // Check the connection status of the integrated WizFi360 module.
  if(WiFi.status() == WL_NO_SHIELD){
    Serial.println("Error: The WizFi360 module is not found!");
    while (true);
  }
  // Attempt to connect to the WiFi network:
  while(status != WL_CONNECTED){
    Serial.print("Attempting to connect to the given network (SSID): "); Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid, pass);
  }

  // If connected to the network successfully:
  Serial.println("WizFi360 module connected to the network successfully!");

  // Initialize the DHT22 sensor.
  dht.begin();

  delay(5000);
}

void loop(){
  get_detection_results();
  collect_weather_data();

  // If the prediction (detection) results are received from the OpenMV Cam H7 successfully, 
  // make an HTTP Get request to the given web application to inform the user of the detection results and the collected weather data via WhatsApp.
  if(OpenMV_data != ""){
    make_a_get_request(OpenMV_data);
    OpenMV_data = "";
  }
}

void make_a_get_request(String detection){
  // Connect to the web application named poultry_feeder_and_egg_tracker. Change '80' with '443' if you are using SSL connection.
  if (client.connect(server, 80)){
    // If successful:
    Serial.println("\nWizFi360 module connected to the given server successfully!\n");
    delay(2000);
    // Create the query string:
    String query = application 
                   + "?temperature=" + String(temperature)
                   + "&humidity=" + String(humidity)
                   + detection
                 ;
    // Make an HTTP Get request:
    client.println("GET " + query + " HTTP/1.1");
    client.println("Host: 192.168.1.22");
    client.println("Connection: close");
    client.println();
  }else{
    Serial.println("\nError: WizFi360 module cannot connect to the given server! \n");
  }
  delay(150);
  // If there are incoming bytes available, get the response from the web application.
  String response = "\nData transferred successfully!\n";
  while (client.available()) { char c = client.read(); response += c; }
  Serial.println(response);
}

void get_detection_results(){
  // Obtain the detection results from the OpenMV Cam H7.
  if(OpenMV.available() > 0){
    Serial.println("\nTransferred results from the OpenMV Cam H7:");
    OpenMV_data = "";
    OpenMV_data = OpenMV.readString();
    Serial.println(OpenMV_data); 
  }
}

void collect_weather_data(){
  delay(2000);
  humidity = dht.readHumidity();
  temperature = dht.readTemperature(); // Celsius
  // Compute the heat index in Celsius (isFahreheit = false).
  hic = dht.computeHeatIndex(temperature, humidity, false);

  Serial.print(F("\nHumidity: ")); Serial.print(humidity); Serial.println("%");
  Serial.print(F("Temperature: ")); Serial.print(temperature); Serial.println(" C");
  Serial.print("Heat Index: "); Serial.print(hic); Serial.println(" C");
  Serial.println("\n");
}

class.php

PHP
<?php

# Include the Twilio PHP Helper Library.
require_once '/twilio-php-main/src/Twilio/autoload.php'; 
use Twilio\Rest\Client; 

class poultry_feeder{
	public $account, $twilio, $conn, $table;
	
	public function __init__($conn, $table){
		# Define the Twilio account information.
		$this->account = array(	
				"sid" => "[sid]",
				"auth_token" => "[auth_token]",
				"registered_phone" => "+[registered_phone]",
				"verified_phone" => "+14155238886"
		);
		
		# Define the Twilio client object.
		$this->twilio = new Client($this->account["sid"], $this->account["auth_token"]);
		
        # Define the MySQL database server settings.
        $this->conn = $conn;
		$this->table = $table;		
	}

	
	# Send a WhatsApp message from the verified phone to the registered phone.
	public function send_message($text){
		$message = $this->twilio->messages 
                  ->create("whatsapp:".$this->account["registered_phone"],
                           array( 
                               "from" => "whatsapp:".$this->account["verified_phone"],       
                               "body" => $text 
                           ) 
                  ); 
 
        echo '<br><br>WhatsApp Message Send...';	
	}
	
	// Database -> Insert Data:
	public function insert_new_data($d1, $d2, $d3, $d4, $d5){
		$sql = "INSERT INTO `$this->table`(`date`, `temperature`, `humidity`, `egg_count`, `feeder_status`) VALUES ('$d1', '$d2', '$d3', '$d4', '$d5')";
		if(mysqli_query($this->conn, $sql)){ return true; }
		else{ return false; }
	}

	// Database -> Create Table
	public function database_create_table(){
		// Create a new database table.
		$sql_create = "CREATE TABLE `$this->table`(		
							id int AUTO_INCREMENT PRIMARY KEY NOT NULL,
							`date` varchar(255) NOT NULL,
							temperature varchar(255) NOT NULL,
							humidity varchar(255) NOT NULL,
							egg_count varchar(255) NOT NULL,
							feeder_status varchar(255) NOT NULL
					   );";
		if(mysqli_query($this->conn, $sql_create)) echo("<br><br>Database Table Created Successfully!");
	}
	
	// Obtain the registered data records (results) from the given database table.
	public function obtain_results(){
	    $sql = "SELECT * FROM `$this->table`";
		$result = mysqli_query($this->conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			$data_array = array();
			while($row = mysqli_fetch_assoc($result)){
				array_push($data_array, $row);
			}
			return $data_array;
		}else{
			$no_data = array([
					"date" => "X",	
					"temperature" => "X",
					"humidity" => "X",	
					"egg_count" => "X",	
					"feeder_status" => "X"
			]);
			return $no_data;
		}
	} 
}

// Define database and server settings:
$server = array(
	"name" => "localhost",
	"username" => "root",
	"password" => "",
	"database" => "poultry_feeder",
	"table" => "entries"

);

$conn = mysqli_connect($server["name"], $server["username"], $server["password"], $server["database"]);

?>

get_data.php

PHP
<?php 

include_once "assets/class.php";

# Define a new class object named 'feeder'.
$feeder = new poultry_feeder();
$feeder->__init__($conn, $server["table"]);

// Obtain the transferred detection results and weather data from the WizFi360-EVB-Pico.
if(isset($_GET["temperature"]) && isset($_GET["humidity"]) && isset($_GET["egg_count"]) && isset($_GET["feeder_status"])){
	// Insert the received information into the given database table.	
	$date = date("Y/m/d_h:i:s");
	if($feeder->insert_new_data($date, $_GET["temperature"], $_GET["humidity"], $_GET["egg_count"], $_GET["feeder_status"])){
		echo("Data received and saved successfully!");
	}else{
		echo("Database error!");
	}
	// Send the received information via WhatsApp to the registered phone so as to notify the user.
	$feeder->send_message(	" $date\n\n"
	                        ." Object Detection\n Egg Count: "
							.$_GET["egg_count"]
							."\n Feeder Status: "
							.$_GET["feeder_status"]
							."\n\n Weather\n Temperature: "
							.$_GET["temperature"]
							."C\n Humidity: "
							.$_GET["humidity"]."%"
		                 );
}else{
	echo("Waiting Data...");
}

// If requested, create a new database table.
if(isset($_GET["create_table"]) && $_GET["create_table"] == "OK") $feeder->database_create_table();

?>

index.php

PHP
<?php
	include_once "assets/class.php";
	
	# Define a new class object named 'feeder'.
	$feeder = new poultry_feeder();
	$feeder->__init__($conn, $server["table"]);
	
	# Get the registered data records and the total entry number from the given database table.
	$data_array = $feeder->obtain_results();
	$data_total = count($data_array);
    

?>
<!DOCTYPE html>
<html>
<head>
<title>AI-driven Poultry Feeder and Unhatched Egg Tracker</title>

<!--link to index.css-->
<link rel="stylesheet" type="text/css" href="assets/index.css"></link>

<!--link to favicon-->
<link rel="icon" type="image/png" sizes="36x36" href="assets/icon.png">

<!-- link to FontAwesome-->
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v6.1.1/css/all.css">
 
<!-- link to font -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Oswald&display=swap" rel="stylesheet">

</head>
<body>
<?php ini_set('display_errors',1);?> 
<h1><i class="fa-solid fa-dove"></i> AI-driven Poultry Feeder and Unhatched Egg Tracker</h1>
<div class="container">
<section>
<h2>Detection Results:</h2>
<table>
  <tr>
    <th>Date</th>
    <th>Temperature</th>
    <th>Humidity</th>
    <th>Egg Count</th>
    <th>Feeder Status</th>
  </tr>
<?php
  foreach($data_array as $row){
	  if($row["date"] == "X") $data_total = 0;
	  echo '
        <tr>
           <td>'.$row["date"].'</td>
           <td>'.$row["temperature"].'</td>
           <td>'.$row["humidity"].'</td>
           <td>'.$row["egg_count"].'</td>
           <td>'.$row["feeder_status"].'</td>
        </tr>		   
	  ';
  }
?>
</table>
</section>
<section>
<div>
<h2><i class="fa-solid fa-database"></i> Database Status:</h2>
<p>Total Data Records: <span><?php echo $data_total; ?></span></p>
<br>
<p>After registering each new detection result to the database table successfully, the application also informs the user via WhatsApp (Twilio API).</p>
</div>
</section>
</div>
</body>
</html>

index.css

CSS
html{background-color:#eb2e00;font-family: 'Oswald', sans-serif;}
h1{color:#002699;text-align:center;font-weight:bold;user-select:none;}
h2{color:white;font-weight:bold;}
p{line-height:normal;font-weight:bold;color:white;}
table{border-collapse: collapse;width:90%;margin-left:45px;margin-bottom:20px;}
table, td, th{border:2px solid white;color:white;}
td{padding-left:20px;padding-top:5px;padding-bottom:5px;}
th{color:#F3D060;background-color:#A5282C;}

.container{position:relative;background-color:none;width:90%;height:500px;margin:auto;margin-top:40px;}
.container section:nth-of-type(1){position:absolute;background-color:#2E3033;top:0;left:0;width:50%;height:100%;overflow-y:auto;border-radius:35px 0 0 35px;}
.container section:nth-of-type(1) h2{color:#F3D060;padding-left:45px;}
.container section:nth-of-type(2){position:absolute;background-color:#ff5c33;top:0;right:0;width:50%;height:100%;border-radius:0 35px 35px 0;}
.container section:nth-of-type(2) div{position:relative;width:90%;height:100%;background-color:none;overflow-y:auto;margin:auto;}
.container section:nth-of-type(2) span{color:#002699;}

::selection {color: #ff5c33; background: #1a53ff;}
/* Width */
::-webkit-scrollbar {width:10px;height:10px;}
/* Track */
::-webkit-scrollbar-track {background-color:#4d79ff;}
/* Button */
::-webkit-scrollbar-button{background-color:#ffc2b3;height:10px;width:10px;}
::-webkit-scrollbar-button:hover{background-color:white;}
/* Handle */
::-webkit-scrollbar-thumb {background-color: #ff9980;}
::-webkit-scrollbar-thumb:hover {background-color: #ffd6cc;}
/* Corner */
::-webkit-scrollbar-corner{background-color:#4d79ff;}

Credits

Kutluhan Aktar

Kutluhan Aktar

79 projects • 290 followers
Self-Taught Full-Stack Developer | @EdgeImpulse Ambassador | Maker | Independent Researcher

Comments

Related channels and tags
  • Adafruit
  • Animals
  • Autodesk
  • DFRobot
  • Edge Impulse
  • Home Automation
  • Machine Learning & AI
  • OpenMV
  • SparkFun
  • WIZnet