Vision based Sand Depletion Prevention Kit (VSDP)

Vision-based Sand Depletion Prevention Kit (VSDP)

This prototype falls under Challenge 1: Climate Crisis Prevention/Mitigation -> Sand depletion crisis

Why did we build this?

Unsustainable sand mining from riverbeds can have substantial social, environmental, geomorphic, and disastrous impacts on rivers. Humanity's appetite for sand could soar 45 percent within four decades, according to researchers who say unchecked consumption risks environmental damage and shortages of key materials for urban expansion Excessive sand mining can alter the river bed, and force the river to change course, erode banks, and lead to flooding. It also destroys the habitat of aquatic animals and micro-organisms besides affecting groundwater recharge

In the IPC, It is defined under section 3(e) of the mines and mineral development and regulation act, 1957 (mmdr act), and this law has been implemented by the government to prevent illegal mining.

News Article- Source:The Wire

A lot of this applies to India, where sand extraction is a major environmental problem. Both legal and illegal sand mining destabilizes river banks and renders them more prone to floods. The demand for sand is driven by rapid urbanization and a construction boom.

News Article- Source:TheHindu

The petition claimed from media reports that Maharashtra had recorded 1, 39, 706 illegal mining cases between 2013 and 2017 whereas Madhya Pradesh saw a total of 42, 152 cases registered from 2009 to 2015. In Punjab, 92 FIRs were filed every month for illegal sand mining.

Hardware Build

I made an infrastructure connected by LoRaWAN which has edge AI nodes monitoring the sand depletion region. This can highly reduce the illegal mining of sand near the water bed thus helping to achieve the UN's 2030 Agenda of the Sustainable Development Goals (SDGs).

Finally, I received my SenseCAP K1100 - The Sensor Prototype Kit with LoRa® and AI from SEEED.

Solution & Workflow

The system is designed to process the sensor data such as VOC, eCO2, Soil moisture, temperature, and humidity using Grove sensors. The device monitors trespassing humans in the sand depletion region and notifies the server using the gateway. The Edge AI model monitors illegal sand mining using the Vision AI Module. Multiple AI nodes are deployed and are connected to the LoRaWAN gateway that connects to the Helium Server. The Seeed Studio LoRaWAN Dev Kit & LoRaWAN gateway will make this system more robust and easy to implement in real-time.

Training using YOLOv5 on Google Colab

After we have chosen a public dataset, we need to train the dataset. Here we use a Google Colaboratory environment to perform training on the cloud. Furthermore, we use Roboflow API within Colab to easily download our dataset.

Click here to open an already prepared Google Colab workspace, go through the steps mentioned in the workspace and run the code cells one by one.

We collected the data, annotated the Image Data and trained the YOLOv5 to classify excavator, human, truck, and wheel loader.

We will now select the UF2 firmware with the trained “VSDP” models. Copy the UF2 firmware to the GROVEAI drive, and you should see a blinking light on the Vision AI module indicating it is performing the firmware update.

Drag and drop the above-generated files to Grove AI Camera drive.

These took more than 8 hours of training on a Google Colab.

Recreating the project

• Add the Wio Terminal Board Library

Open your Arduino IDE, click on File > Preferences, and copy the below URL to Additional Boards Manager URLs:

https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json

Click on Tools > Board > Board Manager and Search Wio Terminal in the Boards Manager.

• Connect the Grove Vision Ai & Grove Lora-E5 to the Wio Terminal. We will be using a built-in light sensor & buzzer and Grove VOC and eCO2 Gas Sensor (SGP30) & Grove Temp & Humidity Sensor (SHT40).

• Download the entire GitHub repo to your local drive. Install all the added libraries from the repo.
• Upload the following code to Wio Terminal

#include "Seeed_Arduino_GroveAI.h"
#include <Wire.h>
#include "Free_Fonts.h" // Include the header file attached to this sketch

#include "SPI.h"
#include "TFT_eSPI.h"

// Use hardware SPI
TFT_eSPI tft = TFT_eSPI();

unsigned long drawTime = 0;
GroveAI ai(Wire);
uint8_t state = 0;
int xpos =  0, ypos = 40;
void setup()
{
  Wire.begin();
  Serial.begin(115200);
  Serial.println("begin");

  xpos = tft.width() / 2; // Half the screen width
  ypos = 50;

  tft.begin();
  tft.setRotation(3);
  tft.setFreeFont(FSSB18);
  // Numbers, floats and strings can be drawn relative to a datum
  tft.fillRect(0, 0, 320, 240, TFT_WHITE);
  tft.setTextColor(TFT_BLACK, TFT_WHITE);
  tft.setFreeFont(FSS12);
  tft.setTextDatum(TL_DATUM);
  tft.drawString("VDSP", 120, 10, 4); // Font 4 for fast drawing with background

  tft.fillRect(20, 55, 140, 58, TFT_GREEN);
  tft.setTextColor(TFT_BLACK, TFT_GREEN);
  tft.drawString(" SECTOR A ", 25, 75, GFXFF);

  if (ai.begin(ALGO_OBJECT_DETECTION, MODEL_EXT_INDEX_1)) // Object detection and externel model 1 - Truck/Excavator
  {
    Serial.print("Version: ");
    Serial.println(ai.version());
    Serial.print("ID: ");
    Serial.println( ai.id());
    Serial.print("Algo: ");
    Serial.println( ai.algo());
    Serial.print("Model: ");
    Serial.println(ai.model());
    Serial.print("Confidence: ");
    Serial.println(ai.confidence());
    state = 1;
  }

  else
  {
    Serial.println("Algo begin failed.");
  }
}

void loop()
{
  if (state == 1)
  {
    uint32_t tick = millis();
    if (ai.invoke()) // begin invoke
    {
      while (1) // wait for invoking finished
      {
        CMD_STATE_T ret = ai.state();
        if (ret == CMD_STATE_IDLE)
        {
          break;
        }
        delay(20);
      }

      uint8_t len = ai.get_result_len(); // receive how many excavator, truck, and wheel loader detected
      if (len)
      {
        int time1 = millis() - tick;
        //Serial.print("Time consuming: ");
        //Serial.println(time1);
        Serial.println("Excavator, Human, truck, or wheel loader detected");
        //Serial.println(len);
        object_detection_t data;       //get data

        for (int i = 0; i < len; i++)
        {
          //Serial.println("result:detected");
          //Serial.print("Detecting and calculating: ");
          //Serial.println(i+1);
          ai.get_result(i, (uint8_t*)&data, sizeof(object_detection_t)); //get result

          //Serial.print("confidence:");
          //Serial.print(data.confidence);
          //Serial.println();
        }
        tft.fillRect(20, 130, 180, 160, TFT_WHITE);
        tft.setTextDatum(TL_DATUM);
        tft.setTextColor(TFT_RED, TFT_WHITE);
        tft.drawString(" Unauthorized entry", 20, 150, GFXFF);
      }
      else
      {
        Serial.println("No identification");

        tft.fillRect(20, 130, 205   , 160, TFT_WHITE);
        tft.setTextDatum(TL_DATUM);
        tft.setTextColor(TFT_BLACK, TFT_WHITE);
        tft.drawString(" Zone is safe", 20, 150, GFXFF);
      }
    }
    else
    {
      delay(1000);
      Serial.println("Invoke Failed.");
    }
  }

  else
  {
    state == 0;
  }
}Now you can see the detection output on the Serial output as shown. Hold the Grove Vision AI module and point the camera at the people or an image with people. The recognized faces/persons will show up in the web browser in a read box together with a confidence level in percent.

Sensor Data Acquisition

The Sensor data are sent periodically using the Grove - Wio-E5.

• Install all the libraries from the above git repo
• Compile the VSDP.ino file and upload it to Wio Terminal.

Sending Triggers to the LoRaWAN gateway

Enclosure - IP-rated enclosure

Making use of IP rated enclosure will make this product robust. PG7 glands were added to connect the soil moisture sensor and temperature, and humidity sensor (SHT40).

We mounted all the sensors inside the enclosure as shown.

The soil moisture sensor was extended using a 3-core 0.5sqmm wire of 1 meter that can be inserted into the soil.

We used a 5V solar panel to power the board. The Li-ion battery is connected to the solar panel as shown below. Connect the C-type connector to the Wio Terminal to power up. I suggest using the Wio Terminal Chassis Battery(650mAh) which could make it even more compact, which will be used in the upcoming development.

The Wio-terminal & Grove Vision AI is mounted to IP rated enclosure that can be mounted to the tree or a post near the water bodies.

Working

On the success of the product, I'd like to scale it to water bodies in my state and explore the product's performance to achieve a sustainable environment. The basic implementation is shown below.

Thanks to Hackster.io, SEEED Studio, Helium, and other sponsors to make this event happen. I think this project adhered to the 4 judging criteria

----------------------------------------------------------------------------------------------------------------

If you faced any issues in building this project, feel free to ask me. Please do suggest new projects that you want me to do next.

Give a thumbs up if it really helped you and do follow my channel for interesting projects. :)

GitHub - https://github.com/Rahul24-06

Happy to have you subscribed: https://www.youtube.com/c/rahulkhanna24june?sub_confirmation=1

Thanks for reading!