Non-Intrusive Load Monitoring using SensiML & QuickFeather

Load Monitoring

Load monitoring and identification is a method of determining the operation and electrical energy consumption of individual appliances based on the analysis of composite load measured from the main supply in a building. It can supply information such as type of load, electricity consumption detail, and the running conditions of the appliances to both the consumer side and the utility side. The information can be used to formulate load plan strategies for optimal energy utilization and helps to reduce energy usage.

Load monitoring techniques can generally be grouped into intrusive and non-intrusive load monitoring. Intrusive load monitoring provides accurate results and would allow each individual appliance's energy consumption to be communicated to a central hub. However, this method is costly because of the number of equipment to be manufactured and installed. Non-intrusive load monitoring (NILM) also known as energy disaggregation is cost-effective and convenient means of load monitoring since it requires lower equipment due to fewer components to install and smaller space requirements.

Load Monitoring helps to reduce global Greenhouse Gas Emissions!

Around 25% of global greenhouse gas emissions come from the burning of coal, natural gas, and oil for electricity and heat generation. It is the largest single source of global greenhouse gas emissions.

Appliance Load Monitoring (ALM) is very important for consumption and energy savings monitoring. Appliance monitoring information leads to the reduction of energy consumption. Furthermore, knowledge of the exact time an appliance is used can be employed in an energy-consumption optimization system. Such systems could provide recommendations to households for potential savings, by deferring appliance use to a day period when electricity is either cheaper or has a lower carbon footprint. Disaggregated data may also help to identify malfunctioning or inefficient equipments and also provides valuable information to facilitate power system planning, load forecasting, new types of billing procedures, and ultimately reduce energy consumption. Reduction of energy consumption means a reduction of greenhouse gas emissions.

D. Vine, L. Buys, and P. Morris, “The effectiveness of energy feedback for conservation and peak demand: a literature review, ” Open J. Energy Effic., vol. 02, no. 01, pp. 7–15, 2013 showed that active energy data feedback to energy users can reduce about 5-20% energy. Real-time Load Monitoring (LM) is considered a good approach to obtain valuable feedback information and perform energy-saving measures, and implement more effective energy management strategies, such as energy efficiency programs, demand-side management, and peak load shedding.

Load Monitoring ultimatelyhelps to Challenge Climate Change!!!

Why Non-Intrusive Monitoring?

The Intrusive load monitoring system is a metering system that measures the energy consumption of an appliance by connecting power meters to each appliance in the household. Therefore, it requires entering the house, thus the system is referred to as intrusive. The intrusive load monitoring system is comprehensive, systematic, and convenient method of monitoring load. However, the cost of installation and maintenance of measuring devices such as meter or sensors may be too expensive. To adequately monitor appliances, sensors are installed at each appliance to be monitored, and once the appliance recorder receives a sensor message, it immediately records the load data and delivers them to the data centre for further analyses. In addition, the intrusive load monitoring technique is too complicated to implement in an ordinary household without a two-way communication device such as smart meter.

Non-intrusive load monitoring is a convenient means of determining the energy consumption and the state of operation of individual appliances based on analysis of the aggregate load measured by the main power meter in a building. NILM is a process of analysing changes in the voltage and current going into a building and deducing what appliances are used in the building as well as their individual energy consumption. It is called non-intrusive because it does not require intruding into the house or consumer premises when measuring the power consumption of different appliances. Smart meters with NILM technology are used by utility companies to survey the specific uses of electric power in different homes. NILM is considered a low cost alternative to intrusive monitoring techniques. The idea of analysing the power flow to determine household appliances and report on their operating condition started when George W. Hart was collecting and analysing load data as part of a residential photo-voltaic system studied (Ehrhardt-Martinez, 2010). The basic monitoring principle is to recognize a step change in active and reactive power in the total load produced by altering the perating state of the different customer’s appliances.

How NILM Works?

A NILM is designed to monitor an electrical circuit that contains a number of devices (appliances) which switch on and off independently. By a sophisticated analysis of the current and voltage waveforms of the total load, the NILM estimates the number and nature of the individual loads, their individual energy consumption, and other relevant statistics such as time-of-day variations. No access to the individual components is necessary for installing sensors or making measurements. This can provide a very convenient and effective method of gathering load data compared to traditional means of placing sensors on each of the individual components of the load. Traditional load research instrumentation involves complex data-gathering hardware but simple software. The NILM approach is opposite of this, with simple hardware but complex software for signal processing and analysis. Only a single point in the circuit is instrumented, but mathematical algorithms must separate the measured load into separate components. NILM has been performed through machine learning and due to advanchment of machine learning, TinyML and electronics, NILM energy disaggreation can now be performed very easily and acurately even using a cheap microcontroller.

SensiML and QuickFeather Development Kit are Perfect Choices

Current approaches for collecting and labeling Non-Intrusive Load Monitoring (NILM) datasets still rely heavily on a lengthy and error-prone manual inspection of the whole dataset. Consequently, it is still difficult to find fully labeled datasets that could help the research in this field. In an attempt to overcome this situation, a hardware and software platform is required to collect and label NILM sensor data using a graphical user interface where the end-user can supervise the labeling process.

For detecting and identifying the load in real-time a machine learning model need to be developed from the collected data. Applying machine learning through programming required an experienced person in the field for getting a good result. But using a graphical machine learning platform like SensiML can make it very easy even for an inexperienced developer.

SensiML is a SaaS-based tool suite for creating optimized sensor AI algorithms executing on low-power microcontrollers. SensiML Toolkit provides a complete development workflow for data collection, labeling, model generation, and test/validation of embedded AI code with industry-leading code efficiency and memory footprint.

The QuickFeather Development Kit is a small form factor system ideal for enabling the next generation of low-power Machine Learning (ML) capable IoT devices. Unlike other development kits which are based on proprietary hardware and software tools, QuickFeather is based on open source hardware, compatible with the Adafruit Feather form factor, and is built around 100% open source software (including the Symbiflow FPGA Tools).

The QuickFeather is powered by QuickLogic’s EOS™ S3, the first FPGA-enabled Arm Cortex®-M4F MCU to be fully supported with Zephyr RTOS.

SensiML Toolkit with the QuickLogic QuickFeather Development Kit is a perfect combination for data collection, labeling, model generation, and deployment for NILM. SensiML generated model can be directly used to develop firmware for QuickFeather Dev Kit and in this way, we can develop a commercial machine learning capable product using QuickFeather DK and SensiML Toolkit.

Workflow of Our System

Before starting the project let me explain what we are going to make. This will be a PoC project for Non-intrusive Load Monitoring. This project will help to detect the connected/running appliance in our house and we will be able to monitor it from a remote location. As I already explained, for detecting and identifying electrical load automatically Machine Learning is required. QuickFeather Dev Kit will be the heart of our system. QucikFeather board will be used to collect required data from the sensors. We will record and label the data using SensiML Data Capture Lab and then with the help of SensiML Analytics Toolkit we will build an ML model for detecting and identifying the load automatically. We will build the model for few common household appliances only. After building the model we will download it as a library for QuickFeather Dev Kit. We will use this library for developing the firmware for quickfeather board. We will add a OLED display with the quickfeather for locally monitoring the load. We will also connect Node MCU WiFi board with the quickfeather to send data to the cloud. We will use Firebase Real-time Database for data storage. Then we will develop an android application to visualize the data for remote monitoring. The block diagram of the system is shown in the following image.

Getting Started with QuickFeather Dev Kit

Development Environment Setup

For getting started with the QuickFeather Dev Kit first you need to set up your development environment. To do this follow the steps below:

1. Install all the required toolchain by following the guide: https://qorc-sdk.readthedocs.io/en/latest/qorc-setup/quickstart.html

2. Clone this QORC SDK repository using the command or just download it directly from GitHub link.

git clone --recursive https://github.com/QuickLogic-Corp/qorc-sdk

3. Install and configure Eclipse for use with QORC SDK sample project by following the guide: https://github.com/QuickLogic-Corp/qorc-sdk/blob/master/using_eclipse.rst

For our project first, we need to develop firmware for the Quickfeather board to record current and voltage waveforms from the sensors. Then, from the recorded data we will develop a machine learning model using SensiML Analytic Studio. We then download the model as a library and use this library to develop the final firmware for the Quickfeather board for real-time monitoring and detecting the load. For the whole process, we will use the Simple Streaming project provided with the QORC SDK. We will modify some files of the project to fit with our sensors and requirements.

Modifying Quickfeather Simple Streaming Interface AI Application Project

This project performs either data collection or recognition based on the build mode. Data collection uses Simple Streaming Interface to connect and stream sensor data to SensiML's Data Capture Lab. Adding external sensors to collect data, analyze and build models is made simple requiring only a sensor configuration API function and data reading function API to be supplied for the new sensor. This project provides an Arduino-friendly Wire interface for easily integrating Arduino sensor code.

In this project, we are going to detect and Identify load from voltage and current waveform of common home appliances. For real-time load identification, we need to train our device with similar data. So, we will record voltage and current for some common electrical home appliances in SensiML Data Capture Lab using Quickfeather board.

We will use the ZMPT101B voltage sensor module for measuring the AC voltage and the ZMCT103C current sensor module for measuring the load current. Both sensors provide analog data. For interfacing the sensors with QuickFeather board we will use ADS1115 I2C analog to digital converter module. We should and will use the same sensors for real-time detection for accurate results.

Though ADS1115 is a 4 channel AD converter we will use two separate ADS1115 for collecting voltage and current. The reason for using two is that when I tried to read two channels simultaneously in the continuous mode from the same module I observed some delay. But for better results and for calculating the real power we need faster data collection. Two separate AD converter modules solved the problem.

The simple streaming interface project provided with the SDK uses the onboard accelerometer sensor for collecting movement data. But we are going to collect different data from different sensors using ADS1115 AD converter module. So, for using this module to collect voltage and current we need to modify the simple streaming app. You need to follow the steps below for doing the required modification:

1. Import the qf_ssi_ai_app application to the Eclipse environment from qorc-sdk directory by following this guide.

2. Download or clone the RobTillaart ADS1115 Arduino library from here.

3. Unzip the library and copy the ADS1X15.h file from the src directory to the inc folder of the application directory (qorc-sdk/qf_apps/qf_ssi_ai_app/inc) and import the file to Eclipse also.

4. Copy the ADS1X15.cpp file from the src directory to the src folder of the application directory (qorc-sdk/qf_apps/qf_ssi_ai_app/src) and import the file to Eclipse also.

5. Add the following lines to ADS1X15.cpp file to define the Arduino delay() function:

extern "C" void HAL_DelayUSec(uint32_t usecs);
void delay(unsigned long ms){
HAL_DelayUSec(ms*1000);
}

6. Remove or comment #include "Arduino.h" from ADS1X15.h file and save it. Quickfeather has no Arduino.h like header.

7. Add the following lines to define two ADS1115 sensors. We are using 2 ADS1115 here for continuous reading of voltage and current at a high speed. The default address of the ADS1115 sensor is 0x48. For using two we need the change the default address for one sensor. For doing so connect the Address pin (ADDR) of the sensor module to VCC. Connecting with vcc will change the i2c address from 0x48 to 0x49.

ADS1115  qorc_ssi_adc_volt(0x48);  //read voltage sensor
ADS1115  qorc_ssi_adc_amps(0x49);  //read current sensor
//MC3635  qorc_ssi_accel;  //////////////////////////

8. Update the function sensor_ssss_configure() by adding the following lines of code in sensor_ssss.cpp to initialize and configure the sensors.

qorc_ssi_adc_volt.begin();
qorc_ssi_adc_volt.setGain(1);     //+/- 4.096V
qorc_ssi_adc_volt.setMode(0);     //1 = single reading, 0 = continious reading
qorc_ssi_adc_volt.setDataRate(7); //maximum data rate
qorc_ssi_adc_volt.readADC(0);     //Continuously read channel 0

qorc_ssi_adc_amps.begin();
qorc_ssi_adc_amps.setGain(1);     //+/- 4.096V
qorc_ssi_adc_amps.setMode(0);     //1 = single reading, 0 = continious reading
qorc_ssi_adc_amps.setDataRate(7); //maximum data rate
qorc_ssi_adc_amps.readADC(0);     //Continuously read channel 0

9. Update the string value definition of json_string_sensor_config in sensor_ssss.cpp for the new sensor added to this project:

const char json_string_sensor_config[] = \
"{"\
"\"sample_rate\":100,"\
"\"samples_per_packet\":6,"\
"\"column_location\":{"\
"  \"Voltage\":0,"
"  \"Current\":1,"
"  \"Power\":2"
"}"\
"}\r\n" ;

10. We need to convert the sensors reading to the appropriate voltage, current, and power. Add the following lines of code to sensor_ssss_acquisition_buffer_ready() function for this purpose:

int16_t adc_voltage = qorc_ssi_adc_volt.getValue(); //get value from channel 0
int16_t adc_current = qorc_ssi_adc_amps.getValue();

int16_t offset_voltage = adc_voltage - 13000;  //offset for zero reading 
int16_t offset_current = adc_current - 13000;  

double converted_voltage = (4.096 / 32768.0) * offset_voltage; //see the lib doc
double converted_current = (4.096 / 32768.0) * offset_current;

int16_t insVoltage = converted_voltage * 40000;  //we need to multiply the voltage by 400 for getting actual voltage, then we scaled the voltage by 100
int16_t insCurrent = converted_current * 10000;  //same 100 x 100, see the sensor datasheet
int16_t insPower = insVoltage*insCurrent;

The Real Power is the actual power consumed by an electrical load. It can be calculated by averaging the product of instantaneous voltage and current waveform values. I tried to calculate the real power by averaging 100 samples per set but I was not successful. It may be for the delay in sensor reading or any other configuration on the program. It works for 20 samples for my case but more samples will give accurate results.

int16_t temp_voltage, temp_current, no_of_sample = 20;
double voltage, current, ins_voltage, ins_current, ins_power = 0;
double real_power = 0;

for(int i=0; i<no_of_sample; i++){
    temp_voltage = qorc_ssi_adc_volt.getValue() - 13000;
    temp_current = qorc_ssi_adc_amps.getValue() - 13000;
    voltage = (4.096 / 32768.0) * temp_voltage;
    current = (4.096 / 32768.0) * temp_current;
    ins_voltage = voltage * 20000; //400
    ins_current = current * 10000;
    ins_power = ins_power + ins_voltage * ins_current;
    }
real_power = ins_power/no_of_sample;
ins_power = 0;

int16_t power = real_power;

11. Update the function sensor_ssss_acquisition_buffer_ready() to read 3 samples (16-bits per channel) into the current datablock. This function returns 1 if datablock is ready for processing, returns 0 otherwise.

int16_t *p_accel_data = (int16_t *)p_dest;

*p_accel_data++ = insVoltage;//accel_data.x;
*p_accel_data++ = insCurrent;//accel_data.y;
*p_accel_data++ = power;     //accel_data.z;

p_dest += 6; // advance datablock pointer to retrieve and store next sensor data

Building the app and uploading the Binary to QuickFeather Board

We have completed all the required modifications for our sensors. Now we need to build the project. The successful building will generate a binary and we will upload the binary to the Quickfeater board.

Our application has built successfully. So let's upload the binary to the QuickFeather board using the command below. I am using Ubuntu here.

Making Hardware Connections

After uploading the binary we are now ready to capture data using QuickFeather Board in SensiML Data Capture Lab. But before starting with Data Capture Lab we need to connect the sensors with the Quickfeather board. As I said earlier I will use two ADS1115 I2C A/D converter modules to connect the current and voltage sensors to the Quickfeather board, we need to connect the components all together according to the following schematic.

The hardware connection without sensors are as follows:

We need to connect the home electrical appliances according to the following connection diagram to record the voltage and current waveforms. The voltage sensor is connected in parallel to the load and the current sensor is connected in series.

I added a socket box with the sensors so that I can easily add or remove any appliance anytime.

Recording data using SensiML Data Capture Lab

1. Now we are ready to use DCL for capturing voltage and current waveform from our appliances. But before starting with DCL we need to change the capture source section in the device plugin file. (the is not mandatory)

2. Run the DCL, import the modified device plugin, and go to the Sensor Properties option by clicking +.

3. Chose the right capture source and sample rate as shown in the above image. Connect the USB TTL converter to the QuickFeather board and plug it into the computer USB. You can get help from this tutorial if you have any confusion.

After successful connection, you will start receiving voltage and current if you have any connected load to the sensors.

4. Choose the right level and start recording. The complete DCL project with all the recorded data is attached in the code section. Take help from that. After recording and segmentation you need to open Analytics Studio to build an ML model.

Train the Model using Analytics Studio

1. Go to https://app.sensiml.cloud/ and log in to your account and open the project.

2. Add a new query following the image below

3. Add a new pipeline and click on OPTIMIZE to build the model

4. After building the model you can explore the result from Explore Model tab

This is the result of the model I built:

The confusion matrix of the model:

5. From the Test Model tab you can test your model for any sample or for all by selecting the check box and then clicking Compute Accuracy button.

6. By clicking the RESULTS you can see the confusion matrix for any specifci type:

7. If you are satisfied with the acuracy then you can download the Knowledge Pack from Download Model tab. The Knowledge Pack can be downloaded in binary form or as a library. We will download it as a library. We will use it in our final code to develop the firmware for Quickfeather for real-time load detection. Set all parameters as follows for downloading Knowledge Pack.

Building the code with the Knowledge Pack

1. Unzip the downloaded Knowledge Pack and copy all three folders from inside the knowledgepack_project directory

2. Paste these inside the knowledgepack folder of the project directory (qorc-sdk/qf_apps/qf_ssi_ai_app/knowledgepack).

3. From the Eclipse open the app_config.h file and set RECOGNITION as 1 and save.

/*######################## FIRMAWARE MODE SETTINGS  ################################*/

#define S3AI_FIRMWARE_IS_COLLECTION  0	   /* Enable sensor data collection       */
#define S3AI_FIRMWARE_IS_RECOGNITION 1	   /* Enable knowledgepack recognition    */

4. Build the project and upload the binary to the Quickfeather board again.

We are ready to detect the load automatically by the Quickfeather board. Connect the USB TTL module to USB and observe the result in the serial monitor.

Adding Display to Show the Class

In this step, I will add an OLED display to show the Class (Appliance name) detected by the QuickFeather board. I will use a generic 0.96 inch I2C OLED. As this is an i2c display so the connection is very simple. See the schematic below for clarification. SeeedStudio i2c OLED display library with some modifications will be used to add the display support to the QuickFeather board. The modified library is attached.

1. Add the SeeedOLED library to your Eclipse project.

2. Add the following lines to sensor_ssss_configure() function to initialize the OLED display.

SeeedOled.init();  //initialze SEEED OLED display
SeeedOled.clearDisplay();
SeeedOled.setNormalDisplay();
SeeedOled.setPageMode();

3. Create the following new function at the end of the sensor_ssss.cpp file

void display_class_oled(int mod, int clas)
{
    //SeeedOled.clearDisplay();
    SeeedOled.setTextXY(1, 0);         //Set the cursor to Xth Page, Yth Column
    SeeedOled.putString("      NILM");
    SeeedOled.setTextXY(3, 0);         //Set the cursor to Xth Page, Yth Column
    SeeedOled.putString(" Model No: ");
    SeeedOled.putNumber(mod);           //Print number
    SeeedOled.setTextXY(4, 0);
    SeeedOled.putString(" Class: ");
    SeeedOled.putNumber(clas);
    SeeedOled.setTextXY(5, 4);
    if(clas==1)
    	SeeedOled.putString("  Beater");
    else if(clas==2)
        SeeedOled.putString("  Drier");
    else if(clas==3)
        SeeedOled.putString("  Heater");
    else if(clas==4)
        SeeedOled.putString("  Iron");
    else if(clas==5)
        SeeedOled.putString("  Lamp");
}

Class and device name should be according to your ML Model.

4. Add the function prototype to the sensor_ssss.h file

extern void display_class_oled(int mod, int clas);

5. Call the function from sml_output_serial() function of sml_output.c file. This function is responsible to show the class in serial monitor.

display_class_oled((int)model, (int)classification);

Also, add the header file #include "sensor_ssss.h" to this file.

6. Build the project and upload the binary to the QuickFeather Dev Kit. Now you can see the output in the display without connecting it to the computer.

Watch the demo video:

Sending Classification Data to Firebase Real-Time Database

In this step, I will show how to upload the classification result to Google Firebase. As QuickFeather Dev Kit has no internet connectivity we will use ESP8266 Node MCU for sending data to the cloud. Node MCU will receive the data from QuickFeather using UART. Below is the connection with the Node MCU.

We don't need to change anything in QuickFeather firmware. We only need to develop a code for ESP8266 Node MCU for sending the received serial data to Firebase. We will develop the code using Arduino Environment.

1. Download the ArduinoFirebase library from here. Read the library doc to check the other dependencies required.

2. Setup Firebase Real-time Database from here (https://firebase.google.com/). Follow this excellent guide (https://www.hackster.io/pulasthi-nanayakkara/connecting-esp8266-to-firebase-to-send-receive-data-4adf66) to configure your own database and Arduino environment.

3. Add your own credentials in the attached code and upload it to Node MCU.

Put your data inside the double quote:

// Set these to run example.
#define FIREBASE_HOST "your firebase url here"
#define FIREBASE_AUTH "you firebase auth here"
#define WIFI_SSID "your wifi network name here"
#define WIFI_PASSWORD "your wifi password here"

The following function receives data from the QuickFeather:

void serial_read() {
  while (Serial.available()) {    
    // get the new byte:
    char inChar = (char)Serial.read();     
    // if the incoming character is a newline, set a flag
    // so the main loop can do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
    else
    // add it to the inputString:  
      inputString += inChar;
  }
}

The appliance class is retrieved from the string using the following lines of code:

if (stringComplete) {
       //Serial.println(inputString);  ////{"ModelNumber":0,"Classification":4}
       int first_colon = inputString.indexOf(":"); //get the first colon in the received string
       int second_colon = inputString.indexOf(":", first_colon + 1);  //get the second colon in the received string
       //Serial.println(first_colon);
       //Serial.println(second_colon);
       model_number = inputString.substring((first_colon + 1), (first_colon + 1) + 1); //get 1 character after first colon which is model number in our case
       classification = inputString.substring((second_colon + 1), (second_colon + 1) + 1); //get 1 character after second colon which is class number in our case
       //Serial.println(model_number);
       //Serial.println(classification);
       //Serial.println(model_number.toInt() + classification.toInt());
       inputString = "";
       stringComplete = false;
    }

Then the data is uploaded to firebase using the following code snippet:

// set string value
  Firebase.setString("ModelNumber", model_number); //sends model number as string
  // handle error
  if (Firebase.failed()) {
      //Serial.print("setting /message failed:");
      //Serial.println(Firebase.error());  
      return;
  }
// set string value
  Firebase.setString("Classification", classification);  //sends class number as string
  // handle error
  if (Firebase.failed()) {
      //Serial.print("setting /message failed:");
      //Serial.println(Firebase.error());  
      return;
  }
Firebase.setInt("ModelNumberI", model_number.toInt()); //sends model number as intger
  // handle error
  if (Firebase.failed()) {
      //Serial.print("setting /message failed:");
      //Serial.println(Firebase.error());  
      return;
  }

4. Connect the Node MCU to QuickFeather board using UART (TX to RX and RX to TX with a common ground). Follow the schematic above.

5. Power up your circuit and give a look at your Firebase database. You should receive the data as follows:

Making an Android App for monitoring from Smartphone

If we want to monitor the connected load remotely from mobile phone we will be required a smartphone application that can read data from our Firebase URL. I developed a simple demo application for this purpose. I made the app using MIT App Inventor. MIT App Inventor is a nice platform for developing Android apps without any coding.

The above image is a screenshot of the MIT working environment. This environment is for GUI design. The interaction and activity are defined in the Block editor. The following image is a screenshot of the block editor.

This is the full block of my app:

Read the documentation here for details on how to work with Appinventor and Firebase.

https://rominirani.com/tutorial-mit-app-inventor-firebase-4be95051c325

Screenshots of my App:

I attached the complete source code (MILN.aia) in the code section. You can modify it according to your requirement.

Final Words

This PoC project can be used in IoT-based Smart Grid Application, IoT Smart Metering, and Load Monitoring. Utilizing this project in those ways can help to save energy and reduce greenhouse gas emissions.