Save Water

Water is one of the most important substances on earth. All plants and animals must have water to survive. If there was no water there would be no life on earth. Apart from drinking it to survive, people have many other uses for water. With growing population rates and such a small percentage of all the water on Earth fit for consumption, it only makes sense that we must preserve and conserve this precious resource.

Water conservation means using our limited water supply wisely and caring for it properly. Since each of us depends on water to sustain life, it is our responsibility to learn more about water conservation. It is up to each and every one of us to conserve water.

Aim/ Objective

I am going to build an AI water usage monitoring system which will detect whether the water is being used or wasted by analyzing real time audio. This is different from existing solutions because we are using AI to predict the water usage. This is useful because, using this system, we will be saving water. This is an interesting way to conserve water.

Hardware Setup

QuickLogic QuickFeather Development Kit

The QuickFeather Development Kit is a small form factor system ideal for enabling the next generation of low-power Machine Learning (ML) capable IoT devices. Unlike other development kits which are based on proprietary hardware and software tools, QuickFeather is based on open source hardware, compatible with the Adafruit Feather form factor, and is built around 100% open source software (including the Symbiflow FPGA Tools).

The QuickFeather is powered by QuickLogic’s EOS™ S3, the first FPGA-enabled Arm Cortex®-M4F MCU to be fully supported with Zephyr RTOS

6V 3.5W Solar Panel

I will be using a 6V 3.5W Solar panel to power the QuickLogic QuickFeather Development Kit.

Configuring your QuickFeather Development Kit

Before you begin with anything, you should flash your QuickFeather Development Kit with the latest data collection firmware for use with SensiML Data Capture Lab. You can download the binary file from here.

In this project, we will be using the Simple Stream - Audio data collection over USB Serial binary file.

You can build your own binary from the data collection source from the qorc github repo found at https://github.com/QuickLogic-Corp/qorc-sdk.

Data Collection firmware is required to record data with Data Capture Lab. Data collection is disabled when running a Knowledge Pack.

Ensure you have Python installed on your system; it is recommended to have Python 3.6 or newer installed.

You will need the TinyFPGA Programmer from QuickLogic to flash your device. When you download the TinyFPGA Programmer you need to use git to clone the repo from the github. Downloading a zip version of the repo can cause unexpected results.

Use git clone to download the TinyFPGA Programmer with git clone --recursive https://github.com/QuickLogic-Corp/TinyFPGA-Programmer-Application.git and then pip3 install tinyfpgab which installs the Python library.

Place the firmware file in the TinyFPGA programmer directory which also contains the tinyfpga-programmer-gui.py. Plug in the device via USB and press the 'Reset' button on the QuickFeather. The LED will flash blue for five seconds. Press the 'User' button while the LED is still flashing rapidly. After pressing the 'User' button the LED will begin to flash green and the rate of flashing will be quite slow like 'breathing'. This means the device is in upload mode. If the LED is not flashing green, then repeat this step.

While the LED is blinking green, program the data collection binary into QuickFeather by running the following command:

python /Your-directory-path-to-TinyFPGA-Programmer/tinyfpga-programmer-gui.py --port COMX --m4app /Your-directory-path-to-binary/quickfeather-audio-data-collection-usb-serial.bin --mode m4

If the firmware file and the tinyfpga-programmer-gui.py are in the same directory, then you can flash your QuickFeather by running this command:

python tinyfpga-programmer-gui.py --port COMX --m4app quickfeather-audio-data-collection-usb-serial.bin --mode m4

Make sure to change the directory to the TinyFPGA programmer's directory in the command prompt before running this command.

COMX is the COM port for the QuickFeather. You can check the port number by going to Ports in Device Manager in the control panel (if you are using Windows OS).

After flashing your firmware, press the 'Reset' button to load the new application. The LED should blink blue for five seconds and then turn off once it's done.

SensiML Data Capture Lab

Data Capture Lab is a full-fledged, time-series sensor data collection and labeling tool that brings a level of automated dataset management developers are accustomed to in programming tools but is sorely missing in edge ML software until now. SensiML’s approach focuses on allowing developers to build datasets as enduring intellectual property (IP) that can be maintained, modified, explored, extended, and exported easily as required. And good data feeding the modeling process translates to good ML inference code as output. [source]

Setting up Data Capture Lab

To get started, create a new account and download the appropriate DCL software. After downloading the software, sign in to your account.

Create a new project and save it in the SensiML projects directory.

After opening your project, click Switch Modes and open Capture Mode.

Device Plugins are a list of properties that describe how the DCL will collect data from your device. For example, the device plugin may contain a list of sample rates that your device supports. This allows the DCL to collect data from any device that has been built to accept the supported parameters below.

You can download the example.SSF file for Simple Streaming Protocol from here. The Data Capture Lab allows you to import Device Plugins via.SSF files through the menu item Edit → Import Device Plugin… Next, you will be able to select your plugin protocol.

Configure your sensor and set the appropriate sampling rate. We will be using the microphone in this project.

The microphone in the QuickLogic QuickFeather Development kit is the Infineon IM69D130 MEMS microphone which has a sensitivity of -36.0 dBFS and the signal-to-noise ratio is 69 dB(A).

Capturing Data

Within the hardware setup found in the right side of the DCL software, set the Capture Method as Live Stream Capture and the connection method as Serial Port.

Select the 'Find Devices' option and click on scan for devices after you have plugged in your QuickFeather Development kit. Select the appropriate COM port and connect your device.

Within the label setup, create a label for the event that you are recording. In this case, the labels are Used, Wasted, and Idle. After this, select metadata for the current recording. I have created a class for metadata and added two values which are Train and Test.

When you are ready, press the record button to capture your data.

After you are finished with the process, switch to Label Explorer mode and select Project explorer. Within that, select the file and ensure that the data represents the labels accurately. To accomplish this, break your data into segments. Repeat the process for all the related files within the project explorer.

After going to File menu and selecting Close File option, you are now ready to generate a model from the captured data using the Analytics Studio.

The Data saved within the DCL is automatically uploaded and stored in the cloud.

Analytics Studio

SensiML Analytics Studio, the core of the SensiML software suite, uses your labeled datasets to rapidly generate efficient inference models using AutoML and an extensive library of edge optimized features and classifiers. Using cloud-based model search, Analytics Studio can transform your labeled raw data into high performance edge algorithms in minutes or hours, not weeks or months as with hand-coding. Analytics Studio uses AutoML to tackle the complexities of machine learning algorithm pre-processing, selection, and tuning without reliance on an expert to define and configure these countless options manually.

Whether a seasoned ML expert or just learning the basics of data science, Analytics Studio offers a tool that can substantially increase your embedded algorithm development productivity. [source]

Training a Model

Go to Analytics Studio and sign in to your account. Select the project which you now created in the Data Capture Lab.

To train a model, we must first tell the Analytics Studio which data we want to use in the form of a Query. This can be done by clicking on the Prepare Data tab and entering a name, session, label, relevant metadata, the sensor(s), and how to plot it. After saving the dataset should appear on the right, and we can see how many segments are in each label.

Pipelines can be constructed by going to the Build Model tab and entering a name, the query that was just created, window size (make it the same size as the capture rate for the sensor), optimization metric (f1-score is the most balanced), and the classifier size, which limits how large the model can be, great for loading onto ROM-constrained chips. Clicking Optimize will go through and build the model, and depending on how large the dataset is, it might take a while to complete.

Generating Knowledge Pack

The final step in this project is to deploy the Machine Learning model to your QuickLogic QuickFeather Development Kit. This can be done by obtaining the Knowledge Pack. For this project, we will be downloading the model in the form of binary. In the Download Model tab, select the pipeline you just optimised with the following settings which can be seen in the image below.

Download the zip file and extract the binary file to the directory that contains the tinyfpga-programmer-gui.py. Follow the same steps used to flash the Simple stream firmware.

Open a serial monitor with a baud rate of 460800 and you will be able to see the classification output of the model.

You can check the Machine Learning Model recognition accuracy by using the SensiML Test App.

I am currently facing issues with downloading the Knowledge pack so I could not make a working video at the moment. I've asked for help in the discussion forum. But I will try my best and upload the video when the knowledge pack is available.

Future updates

• Due to the current lockdown in Sri Lanka, I was unable to purchase an Adafruit Huzzah32 - ESP32 Feather board.

In future, I would like to:

• Use the Adafruit Huzzah32 - ESP32 Feather board with the QuickFeather development kit and turn this POC into an IoT POC.
• Test the Machine Learning model in various situations and feed variable data to the Machine Learning model to increase its accuracy.
• Experiment my Machine Learning model with a higher sampling frequency.
• Improve my project with a display to show water usage.
• Add a solenoid water valve to control the faucet and turn it off when not in use.
• Find a way to automatically charge the Li-ion battery powering the QuickFeather development board.