Distributed high speed flow measurement

Introduction

In this project, we tried to achieve high speed flow measurements (tested at 1kHz), with simultaneous data logging and live preview. We took a distributed approach where components share data using gRPC though standard IP networking.

Disclaimer

With this article we want to share an internal experiment that we realized in a restricted time frame and with limited resources. This is not finished software that only waits for you to be used. We share it because the findings are interesting, and we think it could benefit others.

Features

• Support for high sampling rate flow measurement (up to 2 KHz on Raspi 4)
• Viewer provides a live sub-sampled view of the data.
• Viewer also provides a full resolution view when the feed is paused
• Data logger can write data at full resolution

Components

We designed 3 services:

• A server, responsible of acquiring the data at high frequency (running on a Raspi)
• A logging client responsible of writing the measurements to a file
• A viewer client capable of visualizing the flow of data with minimum delay.

Note that it is possible to run the different services on one host. For simplicity, we often ran the logger on the raspberry pi used as server.

Hardware setup

The only hardware setup necessary for this project is the wiring of the sensor to the Pi server. While the SFM3019 uses a Molex Duraclik connector, it is also possible to use a JST PH connector. You can decide to modify the connector provided with the SEK or buy a JST PH cable to female pins.

Please note that by using a third party cable the wire colors will not match the ones shown. Using the wrong wiring could damage your sensor !

Once you have your cable, you can connect the sensor to the Pi as follows:

Server setup

The server is running on a Raspberry Pi (version 4 in our case) as it is quite convenient for prototyping. However, the code can be compiled for another platform.

A detailed setup guide is available in the repository (under raspberry-pi-setup/README.md). You can always refer to those instructions if needed. Here we will take a more generic approach.

First, the Raspberry Pi OS needs to be setup. We recommend using the latest Raspberry Pi Imager to flash a version of Raspbian to the microSD card. We used bullseyelite since bookworm had issues with our Raspi 4. In the settings you should enable SSH connection and Wi-Fi connectivity.

Once the creation of the microSD is complete, you should insert it into your Raspberry Pi and let it boot. Before going further you should check that you can ssh to it. Typically, with the following command:

$ ssh user@hostname.local

user, password and host-name are the ones you defined during the setup of the microSD card. In some cases your network will not resolve hostname.local, in this case you will have to determine the IP assigned to your Raspberry Pi instead.

Once you ensured that your device was reachable, we can transfer the required files on the Pi using the scp:

$ scp ../server/raspi_build/flow_reader/flow_reader flow-reader.service user@hostname.local

Then we can enable I2C on the Raspberry Pi. First ssh to your device, then run the following command to access the configuration script:

$ sudo raspi-config

Under "3 Interface Options" and "I5 I2C", you can enable the I2C interface.

Then you should enable the time sync service by running:

$ sudo systemctl enable systemd-time-wait-sync  --now

Finally, you can enable the flow-reader service we transferred earlier using the following command:

$ sudo mv flow-reader.service /etc/systemd/system/ && sudo systemctl daemon-reload && sudo systemctl enable flow-reader.service --now

And voilà ! the server application is now running and will be restarted after any reboot. You can display the application log using the following command:

$ journalctl -t yafsdv

Viewer client setup

The viewer is a node application using electron.js, we unfortunately did not take the time to create an executable, so you will have to set up a node environment.

You will need both node and npm installed. You can check the versions installed using the following:

$ node -v
v16.14.2
$ npm -v
8.7.0

We used a fairly old version of node.js, a more up-to-date version might work fine as well. You should stick to an LTS version though (as suggested in the electron documentation)

Once you installed node, you can install the dependencies from the repository root folder:

$ npm install

Before running the application, you will need to update the parameters required to connect to your server in the data_client.ts file:

readonly server = 'hostname.local:8080';

Once done and saved you can compile and run the application using:

$ npm run compstart

After pressing "connect" you should see the data coming into the sliding window of 10 seconds.

The live data is sub-sampled. The full resolution is available once the live plot is stopped by clicking on disconnect. You are now able to zoom in to the data.

In those screen captures, we see a pretty noisy signal. It should be noted that it is expected since we used a manual valve and a very small flow (around 2 sL/min) for the demonstration. The sensor being very high resolution, the signal to noise ratio is not very good, but this article is definitely not a showcase about the sensor capabilities.

Logger client setup

As the logger client is based on gRPC and Protobuf, you first need to set up these two requirements. gRPC conveniently bundles both together so that they can be installed in one swoop. It is not that simple however, because we have to build from source by using cmake and make. cmake is a tool that prepares makefiles. Makefiles contain a set of tasks that are required to build an application. These instructions can be used by make to compile the code and link all needed libraries.

Set up gRPC with Protobuf

The following is a short guide with only the essential steps to build and install gRPC on Linux. For more in-depth instructions, feel free to follow the gRPC Quickstart guide instead. The most important thing is that you have gRPC set up with Protobuf before attempting to build the logger.

1. Install cmake and make sure that you have version 3.13 or later:

$ sudo apt install cmake
$ cmake --version
cmake version 3.16.3

2. Install the required build tools:

$ sudo apt install build-essential autoconf libtool pkg-config

3. Clone the gRPC repository:

$ git clone --recurse-submodules -b v1.62.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc

4. Build and install gRPC and Protobuf locally (this can take a few minutes):

$ cd grpc
$ mkdir -p cmake/build
$ pushd cmake/build
$ cmake -DgRPC_INSTALL=ON \
        -DgRPC_BUILD_TESTS=OFF \
        -DCMAKE_INSTALL_PREFIX=~/.local \
        ../..
$ make -j 4
$ make install
$ popd

If you want you can change the installation directory by modifying the CMAKE_INSTALL_PREFIX variable. It is however strongly encouraged to install gRPC locally.

5. Make sure that the local bin folder (or whatever you set as the installation path in the step before) is added to your path variable:

$ export PATH="~/.local/bin:$PATH"

Set up logger

Now that we have installed gRPC and Protobuf, we can navigate to the logging-client folder and start building our logger in the same way:

$ cd logging-client
$ mkdir -p cmake/build
$ pushd cmake/build
$ cmake ../..
$ make
$ popd

As a result we should have a successfully built executable for the logging client in the cmake/build folder.

The logger can now be started with a simple command from the cmake/build directory. Same as with the viewer client, you need to provide the address of the Raspberry Pi on which the server is running. Additionally, you have to configure the file to which the client should append the measurements to. If everything is set up correctly, you should see a message telling you that the logger was able to register successfully:

$ ./logging_client -server_addr hostname.local:8080 -log_file log.txt
Successfully registered with client ID: 1

Logging performance

Except for issues in the communication channel, the logged data should exactly reflect the data acquired by the server with the sensor. Keeping in mind that the server is trying to fetch the data 'as fast as possible', we should see a frequency around 2.2KHz according to the datasheet of the sensor.

On a log file of around 5 minutes we get the following stats for the interval between 2 measurements:

Once we remove the single odd outlier of 7.5s (for readability reasons), we get the following plot:

Overall, the sampling performance is around the expected frequency. Most of the time, the server is pulling as fast as the sensor can answer. Outliers are likely created when the OS handles some interrupts and the pulling gets delayed.

Findings

gRPC in combination with Protobuf is a very powerful tool that can easily handle the high speed flow measurements that we tested over a network. We are far from the limit. We can likely improve the performance by optimizing the packet size compared to the number of message sent.

The viewer client, which was our first application using Electron.JS and Typescript, was also a nice surprise once we passed the initial struggle of the TS on-boarding.

Known issues

The Electron Viewer has a very short lifespan and gets overwhelmed quickly. It starts lagging behind and struggles to keep up with the flow of incoming data. Data-points are shuffled and big gaps appear.

The consistency of the data acquisition by the server should be reviewed as a gap of 7.5s would not be acceptable in a production system.

In general, the performance of each service could be studied more in depth to understand their impact better.