Protect Forests with AI

A forest is considered to be a carbon sink if it absorbs more carbon from the atmosphere than it releases.

Forests are a stabilising force for the climate. They regulate ecosystems, protect biodiversity, play an integral part in the carbon cycle, support livelihoods, and supply goods and services that can drive sustainable growth.

Forests’ role in climate change is two-fold. They act as both a cause and a solution for greenhouse gas emissions. Around 25% of global emissions come from the land sector, the second largest source of greenhouse gas emissions after the energy sector. About half of these (5-10 GtCO2e annually) comes from deforestation and forest degradation.

Forests are also one of the most important solutions to addressing the effects of climate change. Approximately 2.6 billion tonnes of carbon dioxide, one-third of the CO2 released from burning fossil fuels, is absorbed by forests every year. Estimates show that nearly two billion hectares of degraded land across the world – an area the size of South America – offer opportunities for restoration. Increasing and maintaining forests is therefore an essential solution to climate change. [source]

Forests are affected by most commonly affected by wildfires and deforestation.

Wildfire is an uncontrolled fire in an area of vegetation in rural or urban areas, whereas bushfire refers to an uncontrollable fire in bushland. Bushfire/ wildfire can rapidly burn millions of acres of land and can destroy everything - trees, homes, animals and humans - in their way.

We usually hear about California's wildfires. Wildfires happen in other states and countries as well. As of September 25, 2020:

• Over 4.4 million acres have burned across 10 states, 3.3 million in California alone.
• Dozens have died.
• Over 4, 000 structures have been destroyed.
• Almost half a million people have fled their homes. Australia is very prone to menacing bushfires.

From September 2019 until March 2020, when the final fire was extinguished, Australia had one of the worst bushfires in history. 2019 had been the hottest record year for Australia, with the bushfire season starting in June 2019. This caused mass damage all around the country with fires in each state and territory. As of Jan 06, 2020:

• There were about 136 fires burning across New South Wales, Australia on Monday 6 January.
• Over 480 million animals died across New South Wales.
• In total, over 15 million acres have been burned across the country’s six states.
• Officials say 24 people have lost their lives nationwide this fire season.

Hotter weather, a result of climate change, makes forests and vegetation drier and therefore more prone to burning. As a result, the average wildfire season is three and a half months longer than it was a few decades ago, and the number of annual large fires in the West has tripled. Sources say that wildfires will get even more worse in the future.

Climate change may be the reason for worse wildfires/ bushfires, but this disaster also contributes to climate change with uncontrollable greenhouse emissions. Carbon monoxide and carbon dioxide are released in dangerously abundant quantities and this is harmful for both humans and the environment.

Illegal logging and related trade occurs when timber is harvested, transported, processed, bought or sold in violation of national or sub-national laws. It can also happen when forests are cleared for plantations such as oil palm. Illegal logging exists because of increasing demand for timber, paper and derivative products, including packaging.

Illegal logging not only leaves an obvious mark of destruction on forests – gaping holes where ancient trees once stood – it strips the economic livelihood of local communities and responsible companies. [source]

The environmental effects of illegal logging include forest degradation, the loss of biodiversity and the emission of greenhouse gases, which leads to global climate change.

Aim/ Objective

The Aim/ Objective of this project is to detect wildfires and illegal logging by analyzing real time audio using the QuickLogic QuickFeather Development kit and SensiML Analytics Toolkit.

The QuickFeather board will be powered by a rechargeable battery - Li-ion.

Hardware setup

QuickLogic QuickFeather Development Kit

The QuickFeather Development Kit is a small form factor system ideal for enabling the next generation of low-power Machine Learning (ML) capable IoT devices. Unlike other development kits which are based on proprietary hardware and software tools, QuickFeather is based on open source hardware, compatible with the Adafruit Feather form factor, and is built around 100% open source software (including the Symbiflow FPGA Tools).

The QuickFeather is powered by QuickLogic’s EOS™ S3, the first FPGA-enabled Arm Cortex®-M4F MCU to be fully supported with Zephyr RTOS

6V 3.5W Solar Panel

I will be using a 6V 3.5W Solar panel to power the QuickLogic QuickFeather Development Kit.

Configuring your QuickFeather Development Kit

Before you begin with anything, you should flash your QuickFeather Development Kit with the latest data collection firmware for use with SensiML Data Capture Lab. You can download the binary file from here.

In this project, we will be using the Simple stream - Audio data collection binary file.

You can build your own binary from the data collection source from the qorc github repo found at https://github.com/QuickLogic-Corp/qorc-sdk.

Data Collection firmware is required to record data with Data Capture Lab. Data collection is disabled when running a Knowledge Pack.

Ensure you have Python installed on your system; it is recommended to have Python 3.6 or newer installed.

You will need the TinyFPGA Programmer from QuickLogic to flash your device. When you download the TinyFPGA Programmer you need to use git to clone the repo from the github. Downloading a zip version of the repo can cause unexpected results.

Use git clone to download the TinyFPGA Programmer with git clone --recursive https://github.com/QuickLogic-Corp/TinyFPGA-Programmer-Application.git and then pip3 install tinyfpgab which installs the Python library.

Place the firmware file in the TinyFPGA programmer directory which also contains the tinyfpga-programmer-gui.py. Plug in the device via USB and press the 'Reset' button on the QuickFeather. The LED will flash blue for five seconds. Press the 'User' button while the LED is still flashing rapidly. After pressing the 'User' button the LED will begin to flash green and the rate of flashing will be quite slow like 'breathing'. This means the device is in upload mode. If the LED is not flashing green, then repeat this step.

While the LED is blinking green, program the data collection binary into QuickFeather by running the following command:

python /Your-directory-path-to-TinyFPGA-Programmer/tinyfpga-programmer-gui.py --port COMX --m4app /Your-directory-path-to-binary/quickfeather-audio-data-collection-uart.bin --mode m4

If the firmware file and the tinyfpga-programmer-gui.py are in the same directory, then you can flash your QuickFeather by running this command:

python tinyfpga-programmer-gui.py --port COMX --m4app quickfeather-audio-data-collection-uart.bin --mode m4

Make sure to change the directory to the TinyFPGA programmer's directory in the command prompt before running this command.

COMX is the COM port for the QuickFeather. You can check the port number by going to Ports in Device Manager in the control panel (if you are using Windows OS).

After flashing your firmware, press the 'Reset' button to load the new application. The LED should blink blue for five seconds and then turn off once it's done.

SensiML Data Capture Lab

Data Capture Lab is a full-fledged, time-series sensor data collection and labeling tool that brings a level of automated dataset management developers are accustomed to in programming tools but is sorely missing in edge ML software until now. SensiML’s approach focuses on allowing developers to build datasets as enduring intellectual property (IP) that can be maintained, modified, explored, extended, and exported easily as required. And good data feeding the modeling process translates to good ML inference code as output. [source]

Setting up Data Capture Lab

To get started, create a new account and download the appropriate DCL software. After downloading the software, sign in to your account.

Create a new project and save it in the SensiML projects directory.

By default, the Simple Streaming version of the QuickFeather firmware uses the hardware UART. This means that a USB to TTL serial adapter or another Feather/Wing must be used to communicate.

After opening your project, click Switch Modes and open Capture Mode.

Device Plugins are a list of properties that describe how the DCL will collect data from your device. For example, the device plugin may contain a list of sample rates that your device supports. This allows the DCL to collect data from any device that has been built to accept the supported parameters below.

You can download the example.SSF file for Simple Streaming Protocol from here. The Data Capture Lab allows you to import Device Plugins via.SSF files through the menu item Edit → Import Device Plugin… Next, you will be able to select your plugin protocol.

Configure your sensor and set the appropriate sampling rate. We will be using the microphone in this project.

The microphone in the QuickLogic QuickFeather Development kit is the Infineon IM69D130 MEMS microphone which has a sensitivity of -36.0 dBFS and the signal-to-noise ratio is 69 dB(A).

Capturing Data

Within the hardware setup found in the right side of the DCL software, set the Capture Method as Live Stream Capture and the connection method as Serial Port.

Select the 'Find Devices' option and click on scan for devices after you have plugged in your USB to TTL serial adapter connected to the QuickFeather Development kit. Select the appropriate UART COM port and connect your device.

If it doesn't work initially, try unplugging the converter and plugging it back in, or disconnecting and reconnecting.

Within the label setup, create a label for the event that you are recording. In this case, the labels are Fire, Felling and Normal. After this, select metadata for the current recording. I have created a class for metadata and added two values which are Train and Test.

When you are ready, press Begin Recording to capture your data.

After you are finished with the process, switch to Label Explorer mode and select Project explorer. Within that, select the file and ensure that the data represents the labels accurately. To accomplish this, break your data into segments. Repeat the process for all the related files within the project explorer.

After going to File menu and selecting Close File option, you are now ready to generate a model from the captured data using the Analytics Studio.

The Data saved within the DCL is automatically uploaded and stored in the cloud.

Analytics Studio

SensiML Analytics Studio, the core of the SensiML software suite, uses your labeled datasets to rapidly generate efficient inference models using AutoML and an extensive library of edge optimized features and classifiers. Using cloud-based model search, Analytics Studio can transform your labeled raw data into high performance edge algorithms in minutes or hours, not weeks or months as with hand-coding. Analytics Studio uses AutoML to tackle the complexities of machine learning algorithm pre-processing, selection, and tuning without reliance on an expert to define and configure these countless options manually.

Whether a seasoned ML expert or just learning the basics of data science, Analytics Studio offers a tool that can substantially increase your embedded algorithm development productivity. [source]

Training a Model

Go to Analytics Studio and sign in to your account. Select the project which you now created in the Data Capture Lab.

To train a model, we must first tell the Analytics Studio which data we want to use in the form of a Query. This can be done by clicking on the Prepare Data tab and entering a name, session, label, relevant metadata, the sensor(s), and how to plot it. After saving the dataset should appear on the right, and we can see how many segments are in each label.

Pipelines can be constructed by going to the Build Model tab and entering a name, the query that was just created, window size (make it the same size as the capture rate for the sensor), optimization metric (f1-score is the most balanced), and the classifier size, which limits how large the model can be, great for loading onto ROM-constrained chips. Clicking Optimize will go through and build the model, and depending on how large the dataset is, it might take a while to complete.

Generating Knowledge Pack

The final step in this project is to deploy the Machine Learning model to your QuickLogic QuickFeather Development Kit. This can be done by obtaining the Knowledge Pack. For this project, we will be downloading the model in the form of binary. In the Download Model tab, select the pipeline you just optimised with the following settings which can be seen in the image below.

Download the zip file and extract the binary file to the directory that contains the tinyfpga-programmer-gui.py. Follow the same steps used to flash the Simple stream firmware.

Open a serial monitor with a baud rate of 460800 and you will be able to see the classification output of the model.

You can check the Machine Learning Model recognition accuracy by using the SensiML Test App.

I am currently facing issues with downloading the Knowledge pack so I could not make a working video at the moment. I've asked for help in the discussion forum. But I will try my best and upload the video when the knowledge pack is available.

Future updates

• Due to the ongoing lockdown in Sri Lanka, I was unable to purchase an Adafruit Huzzah32 - ESP32 Feather board.

In future, I would like to:

• Use the Adafruit Huzzah32 - ESP32 Feather board with the QuickFeather development kit and turn this Proof-of-Concept (PoC) into an IoT Proof-of-Concept.
• Test the Machine Learning model in various situations and feed variable data to the Machine Learning model to increase its accuracy.
• Experiment my Machine Learning model with a higher sampling frequency.
• Improve my project with a display to show the status.
• Add a GPS and an alert system to alert and share the location that is in danger.
• Find a way to automatically charge the Li-ion battery powering the QuickFeather development board.