Wio Terminal App Sending Sensor Data to Azure Storage Tables

Code on GitHub:Wio Terminal Sending Sensor Data to Azure Storage Tables

Inspired by Benjamin Cabé’s Blog Connecting the Wio Terminal to Azure IoT, actualized version here, I became aware of the Wio Terminal from Seeed Studio.

It’s a pretty complete device with a 320 x 240 Color LCD Screen, coming in a nice case.

More specifications are:

• Cortex-M4F (ATSAMD51P19) Mcu from Microchip, running at 120 MHz192 KB of RAM, 4MB of Flash
• Wireless connectivity: Realteck RTL8720DN with WiFi and BLE
• microSD card reader
• Light Sensor, accelerometer, infrared emitter, microphone, switches.

After successfully completing Benjamin’s tutorial, I thought that the Wio Terminal would be a good candidate to port my application for writing Sensor Data to the Azure Cloud, which I could already successfully implement on the Beaglebone (Mono C# App Sending Sensor Data to Azure Storage Tables - Hackster.io ) and microcontroller boards from GHI-Electronics, which can be conveniently programmed in C# language, basing on TinyCLR OS, a successor of.NET Microframework. App: AzureDataSender_SC20260.

Many people prefere to send telemetry data with mqtt protocol to IoT services in the Cloud or transfere their data to web based portals which give access to graphical views of the data via Internet browser.

However I went the way to write the data directly into Azure Storage Tables and view the data graphically with a smartphone App (Charts4Azure) or in table form (AzureTabStorClient).

With the App Charts4Azure I can see daily Charts of up to 4 analog sensors and up to 4 On/Off-Sensors on one screen, which makes it easy to see dependencies e.g. of the On/Off-Sensors values from the analog values.

Additionally the created Cloud tables can be displayed and downloaded in table form with the free tool Microsoft Microsoft Azure Storage Explorer on Windows, Linux or Mac.

The data are stored securely and are accessible as long as I want. To enable easy batch deleting of old data, there are automatically created new tables for every new year.

Storing data in Azure Storage Tables is rather inexpensive and in many cases seems to be cheaper in comparison to sending data via mqtt based IoT services.

Transmission is done secure via https (TLS 1.2) and data integrity is ensured by design.

But now let's start:

Get an Azure Storage Account with Account –name and -key

• If you don’t have an Azure subscription, create one for freebefore you begin. ( Learn what you can do with an Azure free account - YouTube )
• Then create a Storage Account. ( Create a storage account - Azure Storage | Microsoft Docs ): On the page Home – Microsoft Azure click on the ‘+’ sign in the left upper corner to create a new resource. In the menue search for Storage Account or select it when already visible. Click on the Create Button. Under Replication select ‘Locally-redundant storage’ if you want to select the cheapest variant. When the storage account is created click on the name of the Account on the ‘Home’ page of the portal. Then select ‘Access keys’ in the left column of the page. Then click on ‘Create Access Keys’ and copy the first Access Key to your clipboard. Store the name of your Storage Account and the Key in a save place on your PC. The name you should note will look like: ‘mynamexyz’, the key will look like: 3J+rsY2Wp6d9dOB/MoN5tXczzLj4aPkhjqxPcJgS2GtJyiCu6pbM/tzKNm3s4UhSdZpAs+R1UtmaAOjg+1r7LA==You will need both to be entered in the configuration file 'config.h' for ‘AzureDatasender_Wio_Terminal’ to savely connect to your Storage Account.

Update the WiFi Chip firmware of Wio Terminal

Before the Application can run on your board, the firmware of the WiFi chip of the Wio Terminal has to be updated to the latest version (actually v2.1.1). The procedure is described in detail in the Seeed Studio Wiki and this video tutorial. The procedure is easy and described very clearly. It is strongly recommended to strictly obey the tutorial. Don’t hurry here !! If you do something wrong, you will have a hard time to fight against the wrong WiFi firmware on your board.

Install the Application on the Wio Terminal

The program ‘AzureDatasender_Wio_Terminal’ is written in C/C++. Development was done on VS Code with the PlatformIO extension (PlatformIO IDE - Visual Studio Marketplace ) installed. In the following steps I assume that both are installed and that you have already been able to run a “blinky” or “Hello world” on your Wio Terminal.

To proceed, now download ( RoSchmi/AzureDataSender_Wio_Terminal: Wio Terminal sending telemetry data to Azure Storage Tables (github.com) ) the Code as a ZIP file and unzip it in your working directory of VS Code. Rename the folder ‘AzureDataSender_Wio_terminal-master’ into ‘AzureDataSender_Wio_terminal’. Open the folder in VSCode. Now you have to wait some time until all the included libraries are automatically installed.

Then, just build the application to see what happens:

You will get the error message that the file ‘config_secret.h’ is missing.

Now, in VS Code open the folder ‘include/’, right-click on the file ‘config_secret_template.h’ and rename it into ‘config_secret.h’.

Open the file ‘config_secret.h ’and enter your WiFi Credentials as well as name and key of your Azure Storage Account which you got in the step described before.

Now build the application again. It should now successfully compile and only show several warnings.

Now deploy the application to the Wio Terminal. The program should automatically start, connect to your WiFi router, get the actual time over internet and begin to upload sensor data to your Azure Storage Account. It will automatically create five tables named ‘AnalogValuesX2021’ ‘OnOffx01x2021’, OnOffx02x2021’, ‘OnOffx03x2021’, ‘OnOffx04x2021’.

Be aware that you should use an empty Storage Account to be sure that you don’t change pre-existing tables in your account.

If something isn’t working as expected please doublecheck your WiFi-Credentials and Azure Storage Credentials and have a look in the file ‘include/config.h’. Try to set ‘#define TRANSPORT_PROTOCOL 0’ (http instead of https) and adjust the settings for your time zone (is explained in detail below).

Now, if it is working, it’s time to have a look, what ‘AzureDatasender_Wio_Terminal’ has produced in your Azure Storage Account. You can use the free tool Azure Storage Explorer, which is available for Windows, Mac and Linux or can use the iPhone App AzureTabStorClient to have a smartphone solution.

You can see that ‘AzureDatasender_Wio_Terminal’ writes a new row with sensor data to your database table ‘AnalogValuesX2021’ every minute. If you press and hold one of the three buttons on your Wio Terminal, the application will write a new row in one of the OnOffTable ‘s. If you release the button it writes the next row for the release ( Watch the 4 fields on the bottom of the screen when you press and release, press the blue button in the right lower corner to see a protocol).

Adapt the App to your needs

Nearly all user specific settings can be made in the file include/config.h.

As a first step it is suggested to adapt the Timezone- and Daylightsavingstime settings to your timezone.I think it’s straightforward. The appropriate settings can be found in the given links.

// Set timezoneoffset and daylightsavingtime settings according to your zone
// https://en.wikipedia.org/wiki/Daylight_saving_time_by_country
// https://en.wikipedia.org/wiki/List_of_time_zone_abbreviations
#define TIMEZONE 60 // TimeZone time difference to UTC in minutes
#define DSTOFFSET 60 // Additional DaylightSavingsTime offset in minutes
#define DST_ON_NAME "CEST" // Name of your timezone (summertime)
#define DST_START_MONTH "Mar" // Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov or Dec
#define DST_START_WEEKDAY "Sun" // Sun, Mon, Tue, Wed, Thu, Fri, Sat
#define DST_START_WEEK_OF_MONTH "Last" // Last, First, Second, Third, Fourth
#define DST_START_HOUR 2 // 0 - 23
#define DST_OFF_NAME "CET" // Name of your timezone (not summertime)
#define DST_STOP_MONTH "Mar" // Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov or Dec
#define DST_STOP_WEEKDAY "Sun" // Sun, Tue, Wed, Thu, Fri, Sat
#define DST_STOP_WEEK_OF_MONTH "Last" // Last, First, Second, Third, Fourth
#define DST_STOP_HOUR 3  // 0 – 23

When you have made your timezone settings, just look on the next line. Here you can set the transport protocol, which should normally be set to ‘1’ (https).

#define TRANSPORT_PROTOCOL 1 // 0 = http, 1 = https

If you use https, the correct root certificate of the Azure Storage service (actually: Baltimore Root_Ca) must be present in in the file config.h. If this Root Certificate should change in future, a actualized one has to be included in config.h and the code has to be adapted to use the new Root Certificate.

typedef const char* X509Certificate;
X509Certificate myX509Certificate = baltimore_root_ca;

Then you can change the sendinterval. The analog values are sent according to this interval to the Cloud. For the On/Off tables a new row is sent every time the state changes.

#define SENDINTERVAL_MINUTES 1.0 // Sendinterval in minutes

For normal operation this interval should be set to 10 or 5 minutes. This assures that you don’t produce to many data in the Cloud and shortens the download time when you want to watch your data graphically with the App Charts4Azure on i-Phone or Android Phone.

Another changeable option is the interval to gather readings from the analog sensors.

This setting refers to all analog sensors. Individual changes for special sensors can be made by changing the code.

#define ANALOG_SENSOR_READ_INTERVAL_SECONDS 2 // Analog sensors are read with this interval (limited 1 to 14400)

Then you can change the names of the created Cloud Storage tables according to your needs. Please note that tablenames may only contain alphanumeric characters (no spaces, no underscore etc.)

// Names of tables to be created in your Azure Storage Account
// Per default the names are augmented with the actual year in this App

#define ANALOG_TABLENAME "AnalogValuesX" // Name of the Azure Table to store 4 analog Values max length = 45

#define ON_OFF_TABLENAME_01 "OnOffx01x" // Name of the 1. On/Off Table max length = 45
#define ON_OFF_TABLENAME_02 "OnOffx02x" // Name of the 2. On/Off Table max length = 45
#define ON_OFF_TABLENAME_03 "OnOffx03x" // Name of the 3. On/Off Table max length = 45
#define ON_OFF_TABLENAME_04 "OnOffx04x" // Name of the 4. On/Off Table max length = 45

The Analog sensor labels only affect the names shown on the Wio Terminal screen.

// Labels for sensors to be displayed on Wio Terminal screen (length max 13)
#define ANALOG_SENSOR_01_LABEL "Temperature"
#define ANALOG_SENSOR_02_LABEL "Humidity"
#define ANALOG_SENSOR_03_LABEL "Light"
#define ANALOG_SENSOR_04_LABEL "Movement"

For long term usage of the App you should activate the watchdog and activate the option to reboot after a failed upload. For tests and debugging these option should be inactivated.

#define WORK_WITH_WATCHDOG 1 // 1 = yes, 0 = no, Watchdog is used (1) or not used (0)
#define REBOOT_AFTER_FAILED_UPLOAD 1 // 1 = yes, 0 = no, Because of possible bug in the App or the firmware the App doesn't recover
// after a failed upload, so we should reboot (as long as the bug isn't solved)

In conjunction with this App you should read and store temperature values in °C, don’t use ° Fahrenheit. This App can display the values converted to °Fahrenheit. The phone App ‘Charts4Azure’ can display in °Fahrenheit as well.

# define SENSOR_1_FAHRENHEIT 0 // 1 = yes, 0 = no - Display in Fahrenheit scale
# define SENSOR_2_FAHRENHEIT 0 // 1 = yes, 0 = no - Display in Fahrenheit scale
# define SENSOR_3_FAHRENHEIT 0 // 1 = yes, 0 = no - Display in Fahrenheit scale
# define SENSOR_4_FAHRENHEIT 0 // 1 = yes, 0 = no - Display in Fahrenheit scale

To achieve an appropriate presentation of analog sensor data in the phone App ‘Charts4Azure’ the range of the values should be between -40.0 and 140.0. If your measured values fall outside this range it is recommended to convert them to values fitting in this range. Only change the following settings if you know what you do from analyzing the code and after checking the results.

#define MIN_DATAVALUE -40.0  // Values below are treated as invalid
#define MAX_DATAVALUE 140.0  // Values above are treated as invalid
#define MAGIC_NUMBER_INVALID 999.9 // Invalid values are replaced with this value (should be 999.9) // Not sure if it works with other values than 999.9

Azure Storage Table service presupposes that the System time of the sending device is within certain limits in sync with Azure time. So the real time clock of the Wio Terminal has to be synchronized with an extern time source from time to time. This App supports two ways how this synchronization can be done. The first way is to call a time server via the NTP protocol, the second was is to synchronize the System time with the time value retrieved from the response header which is sent back from Azure after a received Post request. Both ways can be used in conjunction: Normally the way to synchronize from Azure should be activated and NTP only be set to a short interval if there is a very long time interval between the uploads to Azure.

#define NTP_UPDATE_INTERVAL_MINUTES 20 // With this interval sytem time is updated via NTP
// with internet time (is limited to be not below 1 min)
#define UPDATE_TIME_FROM_AZURE_RESPONSE 1 // 1 = yes, 0 = no. SystemTime is updated from the Post response from Azure.
// With this option set, you can set NTP_UPDATE_INTERVAL_MINUTES to a very
// long time, so that it 'never' happens

Diagnostic Settings (not needed for normal operation)

In normal operation the App stores data of up to 4 analog Sensors in one table in the Cloud and level changes of 4 On/Off Sensors in 4 assigned tables in the Cloud (one table for each sensor). The App calculates the cumulated On-times of each sensor over the day and stores these values in the Cloud table in every row.

For diagnostic purposes simulated analog- and On/Off values can be automatically created by ‘AzureDatasender_Wio_Terminal’. Uncomment the line

//#define USE_SIMULATED_SENSORVALUES
// if activated we send simulated values building sinus curves

To activate this function. If you uncomment additionally the line

//#define USE_TEST_VALUES
// Activates sending of test values (see Code in main.cpp)
// if activated we send diagnostic test values, not sinus curves

You can store up to 4 diagnostic values (numbers between -40.0 and 140) in the table which is normally used for analog sensor values (accordingly make changes in the Code).

Handling of outdated values (...good to know)

In this App we use related to the analog values some kind of decoupling between gathering of sensor values and writing of sensor values to the Cloud. The last sampled values are sent to the cloud except they were not actualized within in a certain timespan. If the timespan has expired, a special magic value (999.9) is sent instead of the last measured value, indicating that an actual sensor value is missing. The length of this timespan can be defined in the settings.

#define INVALIDATEINTERVAL_MINUTES 10 // Invalidateinterval in minutes
// (limited to values between 1 - 60)
// (Sensor readings are considered to be invalid if not successsfully
// read within this timespan)

Integration of other Sensors

a) Analog Sensors

• In this App we use a DHT22 Temperatur/Humidity sensor, the integrated light sensor and the integrated accelerator sensor as examples for analog sensor sources.
• Integration of other analog sensors have to be done in the function ReadAnalogSensor(int SensorIndex), which is called many times per second in the Arduino loop(). To integrate another analog sensor just assign the reading from the sensor to the variable ‘theRead’ or assign ‘MAGIC_NUMBER_INVALID’ which means that there is no value available.
• See the following example for the light sensor:

case 2:
{
    // Read the light sensor
    theRead = analogRead(WIO_LIGHT);
    theRead = map(theRead, 0, 1023, 0, 100);
    theRead = theRead < 0 ? 0 : theRead > 100 ? 100 : theRead;
}
break;

a) On/Off Sensors

• In this App we use as examples the three Wio Terminal buttons. The information that a button was pressed/released comes in the callback of the button interrupt

static void button_handler_right()
{
    int state = digitalRead(BUTTON_1);
    DateTime utcNow = sysTime.getTime();
    time_helpers.update(utcNow);
    int timeZoneOffsetUTC = time_helpers.isDST() ? TIMEZONE + DSTOFFSET : TIMEZONE;
    onOffDataContainer.SetNewOnOffValue(2, state == 0, utcNow, timeZoneOffsetUTC);
}

To integrate another sensor in this way just let your sensor produce a callback function and pass the new state to the function onOffDataContainer.SetNewOnOffValue(…), together with the sensor index (1 – 4), the UTC-time and the offset from your local time to UTC.

Another predefined option to integrate another on/off-sensor is to call the sensor from the Arduino loop() like this is done with the integrated diagnostic 'On/Off Switcher' which is normally set inactive in the code. Here is the code which is called in the loop() to find out whether the state has changed. The new state is than passed to the function onOffDataContainer.SetNewOnOffValue(…) as explained before.

// Check if automatic OnOfSwitcher has toggled (used to simulate on/off changes)
// and accordingly change the state of one representation (here index 2) in onOffDataContainer
if (onOffSwitcherWio.hasToggled(dateTimeUTCNow))
{
    bool state = onOffSwitcherWio.GetState();
    time_helpers.update(dateTimeUTCNow);
    int timeZoneOffsetUTC = time_helpers.isDST() ? TIMEZONE + DSTOFFSET : TIMEZONE;
    onOffDataContainer.SetNewOnOffValue(2, state, dateTimeUTCNow, timeZoneOffsetUTC);
}

Some more words about this App and about Azure SDK for embedded C

Through Benjamin Cabé’s Blog Connecting the Wio Terminal to Azure IoT I became aware of the Microsoft Azure SDK for Embedded C, which is actually still in a beta state. I used some libraries of this SDK in my applictation. The libraries can be found in the folder lib/azure-sdk-for-c/azure and are from version 1.1.0-beta.3.

I couldn’t manage to include these libraries through the PlatformIO libdeps function. If you want to update the libraries for future version download the sdk project as.zip file and copy the folders 'core' and 'iot' from GitHub sdk/inc/azure to the PlatformIO project into the folder lib/azure-sdk-for-c/azure/ using the Windows explorer. Then copy all files from sdk/src/azure/core and sdk/src/azure/iot in your Platformio directory into the already existing folders with the same name ('core', 'iot').

Some files which were needed to support Azure Storage Table service and the Wio Terminal in conjunction with the embedded C SDK can be found under lib/roschmi-for-azur-sdk.

I'm a newbie in programming in C/C++ and using the azure-sdk-for-c is new stuff for me. So please excuse my programming style and how I used the functions of the azure-sdk-for-c. Take the code as a way how it can be done, not the way how it should be done.

Now have fun !!

Let me know if you got it working, and don’t hesitate to make suggestions or ask questions.