Energy Monitoring Device with integrated Web App

What is it?

In this project, I’ve created a self-contained portable Energy Monitoring and Analysis Device with an integrated Web App.

How much TV are you watching in a day? How much does that cost you? What are your usage patterns? How much do you save by running your washing machine at non-peak hours instead of peak hours?

Or if you are a business, how much do you use a certain tool in your business? What are the costs involved with operating that piece of machinery? How stable has been voltage around sensitive lab equipment? What is the quality of power that you are receiving at your location? You could use that information to better estimate your product costs, for example, or by doing anomaly detection or predictive maintenance.

This portable energy monitoring device helps you answer those questions. It measures energy parameters of any device/load plugged in:

• Monitors energy parameters (Current (RMS), Voltage (RMS), Power Factor, Line Frequency, Power, Energy consumed, etc) and writes them to a time series database (InfluxDB)
• Resets energy accumulation at midnight every night, so energy accumulation (cumulative energy consumed) is done on a per-day basis.
• Monitors event conditions (over current/power, voltage sag/surge)
• Presents a web dashboard that shows all the current parameters. Clicking on each parameter also fetches and displays the activity for the last 5 minutes.
• Presents historical data for longer periods using a line graph - you can overlay multiple parameters on the same graph to compare them. You can select the interval for the data from a dropdown - 5m, 10m, 30m, 1hr, 3hr, 6hr, 12hr and 24hr

It has been created using a Raspberry Pi 4, Dr. Wattson Energy Monitoring Board V2 and an OLED screen.

ProtoStax Enclosures for Raspberry Pi 3/4/5 and for Dr. Wattson Energy Monitor provide a nice modular enclosure solution for the whole project, including a power inlet and an outlet whose energy consumption parameters can be measured. Plug in your load of interest and start measuring away!

The Raspberry Pi runs a python Flask based app that queries Dr. Wattson every second and uploads the data to an InfluxDB instance. The app also presents a web user interface using HTML, CSS and Javascript to display the current energy metrics in a dashboard. You can click on each metric to see the last 5 minutes of activity. You can also view historical data via charts - last 5 mins, 10 mins, 30 mins, 60 mins, 3h, 6h, 12h and 24h. You can query the data in JSON format directly with REST APIs. The attached OLED screen on the device shows what the key parameters currently are, so you'll get the data at quick glance by looking at the device screen.

This infrastructure also enables any additional data analysis and control to be built on top of it easily. You can use the web app as-is to get a lot of useful information, both current and historic data (upto 24 hours) without any additional coding. Since historic data is stored in a time series database, you can also query it, visualize it (using tools such as Grafana which goes together very well with InfluxDB), and analyze it with additional coding and data analytics.

Here is the architecture diagram.

Using it

I've tried to simplify the process of using the Energy Monitor in your own setup.

First step is putting together the circuitry and assembling it in the enclosure. ProtoStax and Dr. Wattson have enough information in their user manuals, so I'll refer you to those.

Once you have done that, you'll want to enable the Raspberry Pi for headless operations. The current Raspberry Pi Imager makes it fairly simple by allowing you to configure your WiFi network settings and also enabling SSH, etc.

Once you have a working device all assembled and that you can fire up and login to, you'll have to install the software. You'll first need to install git if you don't already have it:

apt install git

You will also need to install InfluxDB2 and set it up. There are a lot of tutorials for this so I'm not going to repeat it here. You'll need to set up an organization and bucket, and also a token for authentication. Note these values.

After that you can clone the software from Github:

git clone https://github.com/upbeatlabs/UpbeatLabs_EnergyMonitor.git
cd UpbeatLabs_EnergyMonitor

Next we'll create a python virtual environment to facilitate installation of required packages, and then use the requirements from the project to install all the dependencies easily in one go:

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Next you'll need to create a config.ini file to contain the credentials for connecting to your InfluxDB instance. A sample config_sample.ini has been provided. You can use that and update it as appropriate. Remember to create a new file config.ini (rather than modigy the config_sample.ini file)

[APP]
ID = <NAME_OF_DEVICE> 
INFLUX_URL = <URL_FOR_INFLUX_DB> 
INFLUX_TOKEN = <INFLUXDB_TOKEN> 
INFLUX_ORG = <INFLUXDB_ORGANIZATION_NAME> 
INFLUX_BUCKET = <NAME_OF_BUCKET>
INFLUX_BUCKET_AUTH = <NAME_OF_AUTH_BUCKET>

if everything works, you should be able to startup the app like so:

cd UpbeatLabs_EnergyMonitor
venv/bin/python app.py

You can also create a service so that the app starts up automatically whenever the Raspberry Pi starts up. The Github README has instructions on how you can do that. I've provided a sample service file for you to modify as appropriate.

If the code is runnning properly, you should be able to open up a browser to your web app. Let's say you named your Raspberry Pi myPi.local. Then you'd open up the URL to http://myPi.local:5000 (by default, the app starts up on port 5000), and you'll be able to see the dashboard and also your current current data! 😊

Code Deep Dive

The code has been written in Python. I've used Flask, which is a lightweight web framework, to act as web server and to present not only the web UI but also the data in JSON format (which the client side Javascript uses to load and display and refresh historical data).

To communicate with Dr. Wattson, I use it's Python library UpbeatLabs_Python_MCP39F521.

I've also installed an InfluxDB2 database as the time series database. InfluxDB's latest version is 3, but that version has a severely limiting restriction of enabling only about 72 hours of data querying (anything more would not be performant and would put a lot of strain on the system). InfluxDB2 works well and I see myself using that over 3 for the foreseeable future (until they fix/address the issues). With InfluxDB2 you have to use Flux queries, which are not the most popular, but I think they're just fine to use!

For communicating with the OLED screen, I use Adafruit's Adafruit_Python_SSD1306. Note that this library is now deprecated, in favor of their CircuitPython libraries, but that doesn't work yet with Raspberry Pi 5/bookworm. I also had to do a couple of tricks with the GPIO python libraries to get things working on bookworm.

I use the event detection pin from Dr. Wattson to get notified when an event triggering condition has occurred and use that to turn on an LED on the display board.

When connecting to the InfluxDB database, I use a config file to store the requisite information (url, token, org, bucket to use), and use the configparser library to read and get the data. Make sure to create a config.ini file containing your own database information!

The MCP39F521 used by Dr. Wattson provides Energy Accumulation, and Dr. Wattson exposes APIs to turn on/off Energy Accumulation. The python app resets Energy Accumulation every mid-night, so we can keep track of energy accumulation for the day.

In order to make sure that this reset is only done once per day, even if the app restarts, I use the shelve python library to keep track of this information on file. If energy accumulation has already been reset for the day, calling the function does nothing.

With Flask, it defines routes, which include both data (in JSON format), as well as rendering client HTML, CSS, Javascript pages. With Flask, by default, html pages are stored under the "templates" directory, and other static assets like CSS and Javascript are stored under the "static" directory.

For the client side, I use the Bootstrap framework to provide a consistent and mobile-friendly responsive interface without much ado. Of course, there was the learning curve of getting familiar with it! 😊

For the dashboard, as well as the metric charts page, I use Chart.js widgets and graphs to display the various items.

For example, I use "donut" charts to display the current, voltage, line frequency, power factor, etc metrics on the dashboard. I also use the doughnutLabel annotation plugin. This allows me to present the current value as a gauge, for a more pleasing visual appearance. Clicking on each dashboard metric also brings up a popup metric chart showing the last 5 minutes of activity. Getting that to work was a little tricky!

For the event conditions (Over Current, Over Power, Voltage Sag and Voltage Surge) as well as the Power Quadrant representation, I used a Scatter Chart with 0, 0 at the center, and 4 annotations representing the four events or four power quadrants.

In the case of events, one or more events can be active, while the power quadrant can be 1 of 4 values. Mousing over each will give additional information - for example, the condition triggering over current is > 0.18 A. This widget therefore presents a lot of information while appearing minimalist. Mousing over and clicking on a data point gives you more information, and the active regions change color and size (bigger red dot).

The metric charts page displays a line graph and has all the various attributes. By default only the current RMS is displayed, but you can click on other attributes to enable/disable them. Thus a single chart can be used for all attributes. I also added the ability to select different time frames from a drop-down. This fetches new data and re-renders the chart. It also intelligently changes the refresh interval - a 24 hour chart will refresh every minute, while a 5 minute chart will refresh every 5 seconds. The javascript invokes the REST API of the backend with suitable query parameters to fetch the data and update the chart. I had to test both sides to make sure everything was working as expected, and had to uncover and fix a few bugs. All part of the process! 😊

It is also possible to set the min/max for the Y-axis scale - otherwise Chart.js does an auto-scaling based on the data at hand. Setting the scale yourself is useful if you are looking for specific patterns and know the min/max values you are watching and render the chart within those.

In order to share the navigation links and header among all the pages, I have a base.html page and all other pages inherit from this page, so if I want to change navigation links, I can do so in only one location.

Thus you see there's quite a bit of stuff happening on the client side as well!

Going Forward

Since the data is in InfluxDB and it supports Python, you can add additional data analytics by writing a separate Python program and querying the database directly and leverage various Python libraries and frameworks for AI and data analytics (for example, generating daily reports).

Of course, you can also make modifications/enhancements to the existing app:

• Add user authentication so a user has to login before they can see the dashboard
• Use different types of charts - bar charts, scatter plots, etc.
• Add support for querying longer periods of data (currently it queries up to 24 hours). Some things to bear in mind when querying large amounts of data. How can you reduce the amount of data points? You can use data decimation in Chart.js to help reduce the number of data points. Or you can do the processing on the server side to reduce it (for example, aggregate data to 5 min intervals to reduce number of data points).
• Add extra analytics and reporting - you can create a new page to display reports
• Add a page for changing configuration settings - for example, the event triggering conditions (these are hard-coded in right now but could be user configurable)

Of course, you can use the solution as-is too! What would the application be for which you are using it?

What type of load are you going to analyze? What solutions will you build using this? Let us know in the comments below! 😊

Happy Making!

Sridhar