Smart Hives: Beehavior Monitoring

In the last few years, the field of IoT and especially smart home applications has seen a near exponential growth in both customizable protocols and out-of-the-box applications that everyone and their mother have in their homes. These applications are often well suited for their environments, e.g. monitoring and controlling a home, but may be less customizable than what some desire.

The Beehavior Monitoring summer project from Nordic Semiconductor spans over 2021 and 2022. The goal was to use Nordic products to create a scalable system that monitors and extracts useful data from a beehive and to showcase how one may use Nordic products in a more general environment one may want to “smartify”.

In addition to displaying how the prototype developed in this project may translate to any kind of environment, it also aims to make quality-of-life improvements for beekeepers in both urban and rural areas. An extensive portion of beekeeping is time invested in monitoring hive health. This investment can be greatly reduced by applying a low-power solution for monitoring, modeling, predicting, and visualizing hive health through Bluetooth LE and LTE communication.

Model overview

The prototype is divided into two sub-systems: one consisting of an easy-to-scale basis system, and one consisting of an expansion system. The basis system contains the most needed sensors, such as a weight scale, an internal sensing unit monitoring temperature and humidity, a data processing unit, and a cloud unit. The expansion system consists of other sensors such as a woodpecker alarm which requires its own hardware and can be added to the basis system where one considers it necessary.

Sensors

The Beehavior Monitoring prototype uses two sensor units. The internal sensors are all part of the Thingy:52, and the external sensor is a BroodMinder-W weight scale. A third sensor unit is the woodpecker alarm, an external sensor, but this is not yet connected to the rest of the system. The following sensors are the only ones that are supported by the prototype this far, but further support for other Bluetooth LE sensors can be added.

Internal sensors - Thingy:52

Nordic’s Thingy:52 prototyping platform is placed inside the hive to measure temperature, humidity, and air pressure. This project is based on standard Thingy:52 firmware, which means the unit works out of the box. If desired, it is also possible to subscribe to additional Thingy-services by adding the UUID in a similar fashion as done in the Beehaviour Monitoring prototype. For instance, the orientation service can be used to trigger an alarm if the hive has been toppled or physically moved.

External sensors - BroodMinder-W scale

BroodMinder’s Bluetooth-based scale was used to track the weight of the hive as well as the temperature outside of the hive. The nRF5340 unit scans for the scale by address. BroodMinder sensors address are “06:09:16” plus the name of the sensor, e.g. “06:09:16:57:01:FD” if the sensor name is “57:01:FD”. When the sensor is found, the nRF53 unit reads the scan response packet from the sensor and turns it into weight and temperature according to Appendix B in the BroodMinder user guide (pdf-download).

The hive is then raised and placed on top of the scale in one end of the hive and a two” by four” plank in the other end, as suggested by Appendix A in the user guide, except for adjusting the measurement to accommodate for the location of the center of mass. The result of this is that the hive is approximately two times heavier than what the graph in the later section displays. The fix is simple, add a line of code to multiply the scale weight by 2.09 before sending the weight data to the cloud.

External sensors - Woodpecker detector

A feature of the nRF5340 is that we can use machine learning on a microprocessor. An idea that came to mind was to record audio from the beehive and use this together with machine learning to determine the health of the beehive. After investigating the possibilities for using audio to determine hive health, we concluded that there was not enough time to develop a proper model for this purpose. Instead, we moved on to another audio-based module: the Beehavior Monitoring Woodpecker Detector.

To utilize machine learning on the nRF5340, we are using Edge Impulse, a platform for machine learning on embedded devices. They provide ways to create new machine learning models, train them with data you collect or existing datasets, and deploy a trained model to an embedded device. The classification is then performed by the on-site device, and the results forwarded at suitable intervals.

The built in PDM-microphone in the thingy:53 can be used for data acquisition therefore the thingy:53 is the perfect match for being the field device that has the pretrained model. With the big battery and plug and play nature of this setup, this is by far the easiest implementation of the woodpecker alarm.

The woodpecker detector machine learning algorithm was created using Edge Impulse studio, a public version of the woodpecker detector project can be seen here.

Processing- and cloud units

The model is based on one nRF9160 Development Kit (DK) acting as a hub. The board has an nRF9160 SiP that supports LTE-M/NB-IoT to communicate with nRF Cloud, and one nRF52840 SoC to communicate over Bluetooth LE. The nRF9160 DK can connect to up to 19 peripherals, which are the Thingy:53s, where each peripheral represents one hive. Although the system can scale to 19 hives, we have only had a field test with two hives available for testing.

The nRF9160 DK turns data received from the peripheral units (Thingy:53s) into JSON-formatted messages, which are sent to the cloud with a timestamp and the name of the peripheral it received it from, for instance, “Hive1”. The nRF9160 DK can also receive messages from the terminal in nRF Cloud, such as “StartScan” to make the unit scan for peripherals.

The Thingy:53s serve two different purposes. Primarily, they perform data preprocessing/edge computing to reduce the amount of data sent from the nRF9160 DK to nRF Cloud. In addition, they act as a central hub that collects data from the internal and external sensors.

The Thingy:53s also feature multi-role properties where it acts as a central for the sensing units such as the Thingy:52 and the BroodMinder scale. At the same time, it is connected to the nRF9160 DK as a peripheral. This way it can receive data from multiple peripheral units, process it and send it to the central unit, which relays it to nRF Cloud. The data is stored and can be accessed through nRF Cloud’s REST API.

The Thingy:53s can either read the advertising data from peripheral units or connect to the units and subscribe to notifications. To read the advertising data, the unit must scan for the peripheral with a chosen interval. By subscribing to notifications, the Thingy:53 needs a received function for each service subscribed to.

Both the connection and the sending of data between the nRF5340 and the nRF9160 are based on the Nordic UART Service library.

Using nRF Cloud and REST API

The last part of the project consists of visualizing the sensor data and developing a platform that can illustrate the gathered data from the hardware modules.

nRF Cloud is Nordic Semiconductor’s IoT cloud service, and is documented on docs.nrfcloud.com. Our nRF9160 device(s) was added to one of our accounts and messages were sent from the device to the cloud. From nRF Cloud we could also send messages from a terminal in the web interface to the device. All messages sent either to or from the device need to be in JSON format.

A small limitation for historical messages is that nRF Cloud saves them for a maximum of 30 days. To circumvent this, the data is regularly taken from nRF Cloud and moved to a locally hosted backend server.

The messages the nRF9160 DK sends to nRF Cloud are accessed using nRF Cloud’s REST API. More specifically, the endpoint used was ListMessages, which is documented here. When retrieving historical data, the endpoint was accessed multiple times until all messages in the wanted time interval had been retrieved.

The implementation of the REST API functions is mainly found in the GitHub repository NordicPlayground/nrf91-pizza. In addition to the functions implemented in the file js/cloud-api.js, an additional function for retrieving more messages was implemented. All the functions are a member of the class NRFCloudAPI which is initialized with the API key of an nRF Cloud account. It is saved as an access token of the class and used to authenticate the account when endpoints are being accessed.

The following code snippet shows the implementation of the function which retrieves older messages.

getOlderMessages(deviceId, msecs_end, msecs_back) {
    const end = new Date(msecs_end);
    const start = new Date(msecs_end - msecs_back);
    const devIdsParam = deviceId ? `&deviceIdentifiers=${deviceId}` : '';
    return this.get(`/messages?inclusiveStart=${start.toISOString()}
        &exclusiveEnd=${end.toISOString()}${devIdsParam}&pageLimit=100`);
}

When getOlderMessages() is called, it is paired with a loop, which ensures that all messages in a valid time interval is fetched. This method has a limitation for the free nRF Cloud account, which is a maximum of 100 calls to ListMessages. This limitation can be worked around by utilizing pageNextToken, which is part of the response schema from ListMessages.

After fetching the messages, wanted data with associated time stamps are acquired, and can be represented in a suitable way on your website. We used Google Charts to plot data.

Picture text: An example for representing recent sensor data in graphs and info boxes.

Picture text: An example for representing historical sensor data in a graph. Empty area represents a time interval with invalid data.

Due to the hard and continual task to keep the web app secure the project needs a near-maintenance-free solution. This is achieved by using GitHub-pages and a backend computer that fetches data from nRF Cloud and pushes it to GitHub. There is no need to expose any ports to the open web. The below figure is a schematic description of this idea.

The web app itself is compiled to static HTML using the Reactjs framework, where data from the hives are displayed. This implantation is not done yet, but it is in the late stages of development.

Optimizing power consumption