The Things Industries(TTI) V3 Azure IoT Connector

Early November last year I published The Things Network V2 Azure IoT Hubs & IoT Central Gateway. That project was about building a The Things Network(TTN)HTTP Application integration. But, due to some "sub-optimal" design decisions I made and TTN infrastructure moving to V3 the project, though a great learning experience was not that useful.

This project is not quite finished, I need to stress test the MQTTNet based core and try provisioning 100s or maybe even 1000's of devices, add the ability to provision individual devices after the Azure webjob has started, add retry code to the Azure IoT Hub DeviceClient, make the application easier to deploy, sort out The Things Industries(TTI) tenant support, extend Azure Digital Twin(via DTDL) support, and improve the validation of the app.settings.json file.

My TTI V2 gateway used Azure Key Vault to store application secrets but I have had feedback that this was difficult to deploy and manage. This version uses an application settings file but I could add support back in if required.

The Connector has a lot of functionality so to keep the length of the project writeup manageable I have assumed the reader is reasonably familiar with TTI/TTN configuration, Azure Webjobs and Azure IoT Services.

This project is an Azure IoT Identity Translation Cloud Gateway which connects LoRaWAN devices attached to The Things Industries(TTI)/The Things Network(TTN) networks to Azure IoT Hub(s) and/or Azure IoT Central.

The Connector uses the Azure Device Provisioning service(DPS) to configure devices in Azure IoT Central and optionally in Azure IoT Hubs. There is also basic support for the Digital Twin Definition Language(DTDL) so Plug 'n' Play(PnP) devices can be "automagically" provisioned in products like Azure IoT Central.

When the Connector webjob is started it enumerates through the applications configured in the app.settings file. For each application a TTN MQTT Data API session (using the MQTTNet ManagedClient library) is established and the application's up,queued,ack,nack and failed MQTT topics subscribed to.

I used the MQTTNet ManagedClient because it simplified the creation and management of MQTT connections by implementing retries, builders for topics, and client configuration etc.

The Azure webjob needs "Always on" enabled so the in memory caching of MQTTnet Connections and the Azure DeviceClients is persistant.

In this sample app.settings file there are two TTI/TTN applications, "application1" (a very original name) and "seeeduinolorawan".

{
  "ProgramSettings": {
    "Applications": {
      "application1": {
        "AzureSettings": {
          "IoTHubConnectionString": "HostName=....azure-devices.net;SharedAccessKeyName=device;SharedAccessKey=Amk...KM=",
        },
        "DTDLModelId": "dtmi:ttnv3connectorclient:FezduinoWisnodeV14x8;4",
        "MQTTAccessKey": "NNSXS.HCY...RYQ",
        "DeviceIntegrationDefault": false,
        "MethodSettings": {
          "Reboot": {
            "Port": 21,
            "Confirmed": true,
            "Priority": "normal",
            "Queue": "push"
          },
          "value_0": {
            "Port": 30,
            "Confirmed": true,
            "Priority": "normal",
            "Queue": "push"
          },
          "value_1": {
            "Port": 30,
            "Confirmed": true,
            "Priority": "normal",
            "Queue": "push"
          },
          "TemperatureOOBAlertMinimumAndMaximum": {
            "Port": 30,
            "Confirmed": true,
            "Priority": "normal",
            "Queue": "push"
          }
        }
      },
      "seeeduinolorawan": {
        "AzureSettings": {
          "DeviceProvisioningServiceSettings": {
            "IdScope": "0ne...DD9",
            "GroupEnrollmentKey": "AtN...g=="
          },
        },
        "DTDLModelId": "dtmi:ttnv3connectorclient:SeeeduinoLoRaWAN4cz;1",
        "MQTTAccessKey": "NNSXS.V44...42A",
        "DeviceIntegrationDefault": true,
      }
    },
    "TheThingsIndustries": {
      "MqttServerName": "eu1.cloud.thethings.industries",
      "MqttClientId": "MQTTClient",
      "MqttAutoReconnectDelay": "00:00:05",
      "Tenant": "...",
      "ApiBaseUrl": "https://....eu1.cloud.thethings.industries/api/v3",
      "ApiKey": "NNSXS.NR7...ZSA",
      "Collaborator": "devmobile",
      "DevicePageSize": 10,
      "DeviceIntegrationDefault": true
    }
  }
}

The Connector application retrieves a paginated list of each application's EndDevices using the Application Manager API.

There are five seeeduinolorawan devices deployed around my house uploading temperature and humidity telemetry to my Azure IoT Central Instance every 5 minutes.

The "application1" project has a couple of GHI ElectronicsFezduinos with RAK811 Wisduino shields, Seeeduino LoRaWAN with GPS and RAK7200 Track Lite devices.

The connector uses Azure Application Insights with configurable logging levels to make it easier to find and fix setup issues and monitor the application in real-time. I have also tried to format the event payloads to make searching easier.

The Things Industries Configuration

The connector uses a series of classes generated with NSwag (with manually added APIKey support) to call TTI Application API and enumerate page by page the EndDevices in each application.

The DevicePageSize setting specifies how many EndDevices will be retrieved in each page. The DeviceIntegrationDefault setting specifies whether the devices will be processed by the connector, unless overidden by an Application or EndDevice setting.

For each application there are an MQTT access key, DeviceIntegrationDefault and DTDLModelId settings. The DeviceIntegrationDefault setting overrides the Connector setting for this TTI Application, and the DTDLModelId is optional and intended for use where all (or most) of the EndDevices in the Application are of the same type.

The Application API Key needs to have write downlink and read uplink traffic enabled.

The Connector supports both uplink and downlink messages with JSON encoded payloads, so uplink and downlink formatters may required. Especially when a custom or vendor supplied formatter is necessary. This TTI configuration can also be overridden in the EndDevice level.

In my examples I use the Cayenne Low Power Payload(LPP) uplink and downlink formatters.

TTI Application Attributes can be used to override app.settings.json file configuration at an Application level.

In the above example a default dtdlmodelid (beware attribute names have to be lowercase) and deviceintegrationdefault values have been configured..

Configuring an Azure IoT Hub

The simplest way to connect EndDevices in a TTI/TTN Application to an Azure IoT Hub is to use a Shared Access Signature(SAS) Device policy.

For this configuration to work the TTI EndDevice IDs and Azure IoT Hub Device IDs will need to be the same (Beware of case sensitivity in Device IDs).

The core code for processing an Azure IoT Hub message

using (message)
{
   DownlinkQueue queue;
   Downlink downlink;

   string payloadText = Encoding.UTF8.GetString(message.GetBytes()).Trim();

'''

   if (!AzureDownlinkMessage.PortTryGet(message.Properties, out byte port))
   {
      _logger.LogWarning("Downlink-Port property is invalid");

   await deviceClient.RejectAsync(message);
   return;
   }

   if (!AzureDownlinkMessage.ConfirmedTryGet(message.Properties, out bool confirmed))
   {
      _logger.LogWarning("Downlink-Confirmed flag is invalid");

      await deviceClient.RejectAsync(message);
      return;
   }

   if (!AzureDownlinkMessage.PriorityTryGet(message.Properties, out DownlinkPriority priority))
   {
      _logger.LogWarning("Downlink-Priority value is invalid");

      await deviceClient.RejectAsync(message);
      return;
   }

   if (!AzureDownlinkMessage.QueueTryGet(message.Properties, out queue))
   {
      _logger.LogWarning("Downlink-Queue value is invalid");

      await deviceClient.RejectAsync(message);
      return;
    }

   downlink = new Downlink()
   {
      Confirmed = confirmed,
      Priority = priority,
      Port = port,
      CorrelationIds = AzureLockToken.Add(message.LockToken),
   };

   // Don't like using exceptions for flow control but can't find a better appraoch
   try
   {
      // Split over multiple lines in an attempt to improve readability. A valid JSON string should start/end with {/} for an object or [/] for an array
      if (!(payloadText.StartsWith("{") && payloadText.EndsWith("}"))
                             &&
         (!(payloadText.StartsWith("[") && payloadText.EndsWith("]"))))
      {
         throw new JsonReaderException();
      }

      downlink.PayloadDecoded = JToken.Parse(payloadText);
   }
   catch (JsonReaderException)
   {
      downlink.PayloadRaw = payloadText;
   }

   _logger.LogInformation("Downlink-IoT Hub DeviceID:{0} MessageID:{2} LockToken:{3} Port:{4} Confirmed:{5} Priority:{6} Queue:{7}",
   receiveMessageHandlerConext.DeviceId,
   message.MessageId,
   message.LockToken,
   downlink.Port,
   downlink.Confirmed,
   downlink.Priority,
   queue);
...
   DownlinkPayload Payload = new DownlinkPayload()
   {
      Downlinks = new List<Downlink>()
      {
         downlink
      }
   };

   // Need to revisit this..
   string downlinktopic = $"v3/{receiveMessageHandlerConext.ApplicationId}@{receiveMessageHandlerConext.TenantId}/devices/{receiveMessageHandlerConext.DeviceId}/down/{JsonConvert.SerializeObject(queue).Trim('"')}";

   var mqttMessage = new MqttApplicationMessageBuilder()
                 .WithTopic(downlinktopic)
                 .WithPayload(JsonConvert.SerializeObject(Payload))
                 .WithAtLeastOnceQoS()
                 .Build();

   await mqttClient.PublishAsync(mqttMessage);
}

For custom applications sending LoRaWAN downlink messages I have had to add some Message properties for specifying how a message is delivered.

There are four message properties

Port - The LoRaWAN port which is used to specify the "type" or "route" of the message. e.g. I have a sensor which listens on Port 10 for messages to set the update rate, reboot the device etc. This property is required.

Confirmed - This specified whether the downlink message delivery should be confirmed. This should be used with care as it increases network traffic and Azure Messages may timeout before they can be delivered, Ack'd, Nack'd or Failed. This property is optional, the default is false

Priority - TTI/TTN supports seven levels of priority. This property is optional, the default is normal

• Lowest
• Low
• BelowNormal
• Normal
• AboveNormal
• High
• Highest

Queue - Whether the message will be added to the end of the downlink queue or replace any existing ones. This property is optional, the default is push.

• Push
• Replace

In a test environment message payloads and properties can be explored with a tool like Azure IoT Explorer.

The Confirmed property controls how the Azure Message delivery status is set. A TTI/TTN downlink correlation property is used to store the Azure Message LockToken so message delivery progress can be updated.

For unconfirmed downlink messages the Azure IoT Hub deviceClient.CompleteAsync method is called when a "queued" message is received by the TTI/TTN. In addition a flag is set on the downlink payload so no further updates of the message status will be sent by TTI/TTN network.

For confirmed downlink messages deviceClient.CompleteAsync is called when an "Ack" is received, deviceClient.AbandonAsync (message put back on the Azure IoT Hub device queue) is called when called when a "Nak" is received, deviceClient.RejectAsync (message deleted from the Azure IoT Hub device queue) when a "failed" is received. For unrecoverable processing failures like missing required message properties deviceClient.RejectAsync is called.

In a test environment message delivery can be tracked with a tool like Azure Device Explorer Twin.

If an Azure IoT Hub message payload contains valid JSON it will be "grafted" into the decoded_payload property of the downlink message, otherwise the message body will be assumed to be Base64 encoded binary data and copied to the frm_payload property of the downlink message

.For LoRaWAN Class A devices which may send messages a few times a day confirmation of messages can be problematic. The Azure IoT Hub message may timeout and be resent (possibly several times) before delivery can be Ack'd, Nak'd or Failed.

I have also noticed (March 2021) that sometimes a confirmed downlink message can be Nak'd then Ack'd.

Azure Device Provisioning Service Configuration

The connector supports the Azure Device Provisioning Service so devices can be provisioned on demand or as part of a multistep build, ship, deliver, deploy process.

The DPS supports the distribution of devices across multiple Azure IoT hubs for load balancing or to reduce latency. The DPS supports device provisioning with X509 certificates and Symmetric Device keys.

X509 Certificate device provisioning is not supported by the Connector.

I have used the DPS with the Connector to provision EndDevices for TTI/TTN applications in different Azure IoT Hubs.

Configuring Azure IoT Central

Azure IoT Central uses the DPS to manage device connections. The device provisioning process starts with the creation of an "Enrollment Group"

The IDScope and one of the Primary or Secondary Shared Access Signature(SAS) keys should be copied into DeviceProvisioningServiceSettings of an Application in the app.settings.json file. I usually set the "Automatically connect devices in this group" flag as part of the "automagic" provisioning process.

Then device templates need to be created and the Enrollment Group mapped to a device group.

The structure of Azure IoT Central Telemetry and Commands is discussed in excruciating detail in my blog here, here, here, here and here.

The TTI/TTN uplink message payload_decoded fields are mapped to telemetry field names. If the Cayenne LPP format is used the property names are based on the LPP standard. The Connector also "tweaks" GPS Location payload to the format required by Azure IoT Central.

The Connector only supports commands with "Queue if offline", and optional Request Payload.

For a simple Command e.g. "Reboot the device now" which has no parameters the Azure IoT Central command message payload contains only an "@" character. For a simple command a TTI/TTN downlink message with an empty raw payload is sent to the EndDevice.

For a more complex Command e.g. "Set the device uplink period to 3600 seconds" which has only a single parameter the decoded_ payload is populated was a single property which has the name set to the contents of the message property "method-name" and the value (with JSON object and array delimiters removed) in the TTI/TTN downlink message sent to the EndDevice.

For the most complex Commands e.g. "Set the minimum alert temperature to 0.0c and the maximum alert temperature to 30.0c" which have more that one parameter the message payload is JSON and it is "grafted" into the decoded_ payload of the TTI/TTN downlink message sent to the EndDevice.

"MethodSettings": {
    "Reboot": {
      "Port": 21,
      "Confirmed": true,
      "Priority": "normal",
      "Queue": "push"
    },
    "value_0": {
      "Port": 30,
      "Confirmed": true,
      "Priority": "normal",
      "Queue": "push"
    },
    "value_1": {
      "Port": 30,
      "Confirmed": true,
      "Priority": "normal",
      "Queue": "push"
    },
    "TemperatureOOBAlertMinimumAndMaximum": {
      "Port": 30,
      "Confirmed": true,
      "Priority": "normal",
      "Queue": "push"
    }
}

The message port, confirmed, priority and queued values for an Azure IoT Central Command are configured in the TTI/TTN Application MethodSettings in the app.settings.json file.

For Azure IoT Central "automagic" provisioning the DTDLModelId has to be copied from the Azure IoT Central Template into the TTI/TTN EndDevice or Application configuration.

The core code for processing an Azure IoT Central Command message

using (message)
{
   DownlinkQueue queue;
   Downlink downlink;

   string payloadText = Encoding.UTF8.GetString(message.GetBytes()).Trim();

   if (message.Properties.ContainsKey("method-name"))
   {
      // Looks like Azure IoT Central message
      string methodName = message.Properties["method-name"];
      if (string.IsNullOrWhiteSpace(methodName))
      {
         _logger.LogWarning("Downlink-DeviceID:{0} MessagedID:{1} method-name property empty", receiveMessageHandlerConext.DeviceId, message.MessageId);

         await deviceClient.RejectAsync(message);
         return;
      }

   // Look up the method settings to get confirmed, port, priority, and queue
   if (!receiveMessageHandlerConext.MethodSettings.TryGetValue(methodName, out MethodSetting methodSetting))
   {
      _logger.LogWarning("Downlink-DeviceID:{0} MessagedID:{1} method-name:{2} has no settings", receiveMessageHandlerConext.DeviceId, message.MessageId, methodName);

      await deviceClient.RejectAsync(message);
      return;
   }

   downlink = new Downlink()
   {
      Confirmed = methodSetting.Confirmed,
      Priority = methodSetting.Priority,
      Port = methodSetting.Port,
      CorrelationIds = AzureLockToken.Add(message.LockToken),
   };

   queue = methodSetting.Queue;

   // Check to see if special case for Azure IoT central command with no request payload
   if (payloadText.CompareTo("@") != 0)
   {
      try
      {
         // Split over multiple lines to improve readability
         if (!(payloadText.StartsWith("{") && payloadText.EndsWith("}"))
                                             &&
            (!(payloadText.StartsWith("[") && payloadText.EndsWith("]"))))
         {
            throw new JsonReaderException();
         }

         downlink.PayloadDecoded = JToken.Parse(payloadText);
      }
      catch (JsonReaderException)
      {
         try
         {
            JToken value = JToken.Parse(payloadText);

            downlink.PayloadDecoded = new JObject(new JProperty(methodName, value));
         }
         catch (JsonReaderException)
         {
            downlink.PayloadDecoded = new JObject(new JProperty(methodName, payloadText));
         }
      }
   }
   else
   {
      downlink.PayloadRaw = "";
   }

   _logger.LogInformation("Downlink-IoT Central DeviceID:{0} MessageID:{2} LockToken:{3} Port:{4} Confirmed:{5} Priority:{6} Queue:{7}",
               receiveMessageHandlerConext.DeviceId,
               message.MessageId,
               message.LockToken,
               downlink.Port,
               downlink.Confirmed,
               downlink.Priority,
               queue);
}
...

   DownlinkPayload Payload = new DownlinkPayload()
   {
      Downlinks = new List<Downlink>()
      {
         downlink
      }
   };

   // Need to revisit this..
   string downlinktopic = $"v3/{receiveMessageHandlerConext.ApplicationId}@{receiveMessageHandlerConext.TenantId}/devices/{receiveMessageHandlerConext.DeviceId}/down/{JsonConvert.SerializeObject(queue).Trim('"')}";

   var mqttMessage = new MqttApplicationMessageBuilder()
                              .WithTopic(downlinktopic)
                              .WithPayload(JsonConvert.SerializeObject(Payload))
                              .WithAtLeastOnceQoS()
                              .Build();

   await mqttClient.PublishAsync(mqttMessage);
}

The consistent naming of Azure IoT Template telemetry and commands is very important.

But wait there's more...

Once you have telemetry data in an Azure IoT Hub you can do interesting things like...

Use Azure IoT Events as an event source for Azure Event Grid

Use Azure IoT hub Message Routing to send telemetry to other endpoints. In the Connector application the LoRaWAN port, TTI/TTN DeviceID, DeviceEUI etc. properties are added to the Azure IoT Hub Telemetry message so they can be used in routing rules.

Use Azure Logic Applications and "plug-ins" like the Twilio Connector to send alert messages to your mobile

Use Azure Stream Analytics and PowerBI for dashboards and reports

Footnote

I have left out lots of information because the project story length was starting to get a bit silly.

If you're interested in more details I have been documenting progress in my blog since November 2020.