Integrating LoRaWAN with AWS IoT services using the RAK811

Introduction

A lot of buzz has been spurred around the usefulness of the LoRaWAN network and how it is going to change the way IoT device talk over long distances for relaying information. Rightly so, the LoRaWAN technology promises both efficiency and long range making it one of the most sought after wireless tech stack in the field of IoT currently.

We have also already seen a lot of projects that use the venerable TTN network to send and receive LoRaWAN packets using their awesome application and network server infrastructure. Our tutorial will further augment this awesome platform by introducing AWS IoT integration for the LoRaWAN nodes and provide a application interface for developers to provide provisioning and analytics services on top of the TTN API by creating a virtual node for all their end points.

We have already seen how our wifi boards (esp32, rtl8711 rtl8195) based boards can be made to talk to the AWS IoT APIs (https://www.hackster.io/naresh-krish/aws-iot-integration-with-rak-creator-pro-board-8ff2e9 )

The tutorial would expect the readers to go through the setup of node red and AWS IoT account before proceeding with the setup.

• Node-red setup: https://www.hackster.io/naresh-krish/node-red-integration-with-rak-wireless-dash-button-4d2efe

• AWS IoT setup: https://www.hackster.io/naresh-krish/aws-iot-integration-with-rak-creator-pro-board-8ff2e9

The hardware:

For this tutorial I will be covering the RAK wireless gateway and LoRaWAN node, namely the RAK831 and the RAK 811. Both the gateway and node will be talking over the 868 mhz bands. However RAK also provides the 915 mhz band based boards for developers in countries like Australia. Be sure to check out their stores for more information.

The Rak 831 is the LoRaWAN gateway which works with a raspberrypi and wifi as the data backhaul.

The RAK 811 is the LoRaWAN node that can connect to sensors and other Arduino style boards for relaying data. It supports a wide array of AT commands for data transmission and also supports both 5v and 3.3v TTL logic for communication, making it quite a versatile board for makers.

I would expect the readers to go through the tutorial explaining the LoraWAN modules in details and their complete setup procedures

RAK 831 and Raspberry Pi 3: https://www.hackster.io/naresh-krish/getting-started-with-the-rak-831-lora-gateway-and-rpi3-e3351d

RAK 811 lora node setup: https://www.hackster.io/naresh-krish/getting-started-with-the-rak811-lora-node-67f157

These two tutorials should get your setup up and running in no time. I keep them updated for any chnages to the driver setup and hence developers hsould be able to replicate the setup with ease.

The Architecture

Lets see how LoraWAN can talk to AWS IoT Services. The AWS IoT services is a collection of cloud services offered by Amazon for provisioning and maintaing data collected from IoT objects. These can be anything from wifi board, to LoraWAN sensor nodes to industrial sensors etc. The AWS IoT services provide a means to store the state of your things as virtual objects called Shadows. Shadows can have attributes which have a one -to -one mapping with the sensors reading on the nodes or other configuration parameters on the board.

To read/write to these attributes, AWS IoT provides two ways to achieve the updates, MQTT and HTTPS. However both need the devices to send the data via wifi over TCP IP. LoraWAN nodes, however, connect to the gateways via LoRA radio and hence need a platform for converting the gateway wifi payload to a meaning full payload and store the data. This is where the TTN networks awesome infrastructure comes into picture.

TTN provides a application and network server infrastructure for LoraWAN nodes to register to and then send/receive uplink/downlink information.

I have provided a more in depth look at this architecture in the tutorial for RAK811 here:

https://www.hackster.io/naresh-krish/getting-started-with-the-rak811-lora-node-67f157

So now we can see the beauty of the TTN network architecture; the LoraWAN node data is now available for us to manipulate via a cloud service. ALso the TTN MQTT APIs also provide a means to get callback messages wherever an uplink or downlink message is sent fto/from the gateways that we have setup. This MQTT API is the connecting link that we will use in the tutorial to trigger some of our workflows to store the information in AWS IoT things attributes.

BUT HOW DO WE MAKE SURE TTN M<QTT PAYLOAD FOR UPLINK AND DNOWLING MESSAGES REACH AWS IoT SERVICES ????

Node-red to the rescue

We have already seen the awesome stuff that we can do with Node-red and some of the cool workflows modules that we have used to connect various sensors to workflow triggers. So in essence you can create a wifi doorbell, that send you an SMS about the identity of the person at your door by sending their pics to a image recognition service....COOOOOLLLLLLL !!!!

Anyways back to the issue at hand. So we know node-red has a node for AWS IoT shadow updates and a node for TTN node uplink/downlink messages. lets connect the two together shall we :)

Install the required node pallette

1) AWS IoT nodes:

• Proceed to the node-red instllation on your raspberry pi or desktop and click on the palette menu in the top right of the screen

• Search for aws-iot and install the second node in palette.

2) TTN nodes:

• Proceed to the pallete menu again

• Search for ttn

• Install both the nodes as shown above.

Setup Your workflow:

• Install the TTN pallete. It’s the second one in the list below (node-red contrib-ttn):

TTN node

• Once installed, this will show up as two nodes in the left pane like so:

• Select the TTN device node. This node listens for specific TTN MQTT messages raised when a device status/operation enters a particular state some examples are “activation”, “up” etc

• Also drag a debug node into the workflow and connect the output of the TTN device node to the debug node input

Your first lora workflow !!!

Configure your nodes:

• Now double click the TTN node1 node, You will be shown this menu:

• Give a name to this node.

• Now from the TTN console chose your app id and note it down;

• Click on the pencil icon near the App, you will be shows a UI like this:

• This UI you can fill the app id with the Application id from TTN console

• Set the broker to asia. (it can be eu or us as well, check your Handler test in the previous message to know what is your app region)

• In the same app console page scroll down till you reach the Access key section, copy the access key and paste it in the Access key test box. And click Update.

• Now you will return to the original config menu. The device ID is the device name you gave to the rak811 node. Put it in the device ID section

• Finally register for an event. An event would usually be either activation or an uplink message. For now you can test with activation by putting the word “activation” inside your event section

• Next click done

• Now you need to complete the previous menu:

• Now connect your rak811 to your computer and try to connect it to the rak831 gateway either by OTAA or ABP.

• Once connected, send a message to the TTN app. You can test with a simple byte like 0 or 1.

• When your message reaches the TTN app, the TTN MQTT service will send a message to the Asia broker which in turn will send a message to this node as it is registered to that broker

TTN Message decoder payload:

Usually when you send message from a LoRA object you will only send some limited bytes. TTN payload decoder service can help you decode this information and store it in a meaningful way:

On how to use the payload functionality, Check out this excellent example provided by TTN:

https://www.thethingsnetwork.org/docs/devices/uno/quick-start.html#monitor--decode-messages

Troubleshooting:

1) Make sure both the RAK831 and RAK811 show up in the TTN console under gateways and devices respectively.

2) Make sure RAK831 gateway shows up as online in the TTN gateways console

3) Make sure that the region you provide in the TTN node config is the same as the Handler region of the device (asia, or eu, or us etc)

4) Configure the rak811 node to connect to the same app key as you input in the node-red TTN node config.

AWS Node setup:

Once installed the nodes will show up like so

Lets go ahead and add a input AWS MQTT node like so:

Connect a debug node to the output of this AWS MQTT node so we can see the messages from the MQTT topic

MQTT input node. Connects to a broker and subscribes to the specified topic. The topic may contain MQTT wildcards.

Outputs an object called msg containing msg.topic, msg.payload, msg.qos and msg.retain.

msg.payload is usually a string, but can also be a binary buffer.

Steps to configure

• Double click the AWS MQTT node. You will see a screen like so:

• Click on the pencil icon to add details about your AWS IoT things.

• Here give a name to your config node. I generally prefer you give the Thing name as you have regstered in AWS IoT dashboard.

• In Type select the Shadow devices as we will receive the information from the Shadow MQTT topics.

• In the clientID, give you ameba device clientID. it canbe anything like AMebaDevice, RAKDevice etc. just make sure your name is unique to the device in question.

• In the endpoint, give your shadow device endpoint:

It will be available in the Menu side bar under Things -> Your thing -> interact

• Now lets enter our certificate information:

Remember we had downloaded the certificates from AWS when creating the TLS certificates for our Thing. Lets copy those files to a location say /root/.awscerts

Once the files are copied, you need to rename them to match the format needed by the AWS IoT Node Red node. The node expects the files to be named as follows:

/root/.awscerts/
                |--YourUniqueClientIdentifier-private.pem.key
                |--YourUniqueClientIdentifier-certificate.pem.crt
                |--root-CA.crt (downloaded earlier)

Where YourUniqueClientIdentifier is the AWS thing name what you put when creating your thing. You can refer to the other tab that you have open.

• Click Update

Now you will return to the previous menu. Here you need to listen to a particular Shadow service MQTT topic called update/accepted.

• lets enter the following in the topic field:

$aws/things/thingName/shadow/update/accepted 

• Click Done

Now Your node should show connected and start getting messages. Now try tp send an update to the update topic as we had programmed in the RAK473 board. A consequent /update/accepted will shows up in node red and you can now parse the MQTT json data and trigger further jobs. For example, here we try to parse the incoming data and send it to twitter

We can also send the data to an influx db backend for storage and analysis:

Connect the TTN and AWS Services together.

Let the magic happen. Connect the two service together as shows. Please see that there is a fucntion node betwen the TTN output and the AWS IoT input so that we can parse TTN notification output and then send only the relevant information to AWS IoT as required by the AWS services.

Lets see what we need to do in the function node. The TTN node is essential a MQTT node subscribed to the action that we mentioned in the node. SO lets subscribe to a node message receive event. Once you change the action in the TTN node to 'up' events you will essentially subscribe to a topic like so:

hello-world/devices/my-uno/up

This topic will be used by the TTN api to send you a MQTT message every time their is an uplink event from your LoraWAN node.

The payload will eseentially look like this. This is nothing but a JSON

{"port":1,"counter":5,"payload_raw":"AQ==","payload_fields":{"led":true},"metadata":{"time":"2016-09-14T14:19:20.272552952Z","frequency":868.1,"modulation":"LORA","data_rate":"SF7BW125","coding_rate":"4/5","gateways":[{"eui":"B827EBFFFE87BD22","timestamp":1960494347,"time":"2016-09-14T14:19:20.258723Z","rssi":-49,"snr":9.5,"rf_chain":1}]}}

Here we will be more interested in the Payload-fields section or the payload raw section:

{
  
{
  

• Payload-fields: This will be the returned value if you have setup a payload function on TTN console. This essentially will convert your bytes received from LoraWAN node to human readable JSON format

• Paylod-raw is a base64 encoded data that you send directly from your LoRa node and is stored as-is on the TTN servers.

SO you need to parse the payload from the TTN node and send the values to the AWS IoT node. Here are the steps to perform

• First [parse the fields in the payload-fields section

• segregate the fields that have a one to one mapping to AWS IoT attributes

• form your mag.payload for giving as input to the AWS IoT node

• form your msg.payload object to reflect the json string as shown below

 msg.payload = {state:{desired: {led: "true/false"}}} 

Once this msg.payload reached the AWS update request, the AWS IoT service will publish a success on the update/accepted or a failure to update/rejected topics. You can follow the tutorial here to know more in depth about these topics and how to interact with them.

https://www.hackster.io/naresh-krish/integrate-rak473-with-aws-iot-shadow-services-7ce9ab

RAK 811 Arduino code.

Here is a sample of the RAK 811 code which you can run on your arduino UNO for sending messages to the TTN application server:

 void setup() { 
//configure Serial1, this could also be a  
//software serial.  
Serial1.begin(115200); 
//configure the mail RX0 and TX0 port on arduino 
Serial.begin(115200); 
} 
//string to hold the response of a command in rak811 
String response = ""; 
//the famous arduino loop function. runs continuosly 
void loop() { 
//try getting the version of the board firmware 
 sendCommand("at+version\r\n"); 
 //set conn config 
 setConnConfig("dev_eui", "your_dev_eui_here"); 
 setConnConfig("app_eui", "your_app_eui_here"); 
 setConnConfig("app_key", "your_app_key_here"); 
 //join the connection 
 sendJoinReq(); 
 //send data too gateway 
 sendData(1,2,"000000000000007F0000000000000000"); 
 delay(3000);  
} 
/** 
* Function to send a command to the  
* lora node and wait for a response 
*/ 
void sendCommand(String atComm){ 
response = ""; 
Serial1.print(atComm); 
 while(Serial1.available()){ 
   char ch = Serial1.read(); 
   response += ch; 
 } 
 Serial.println(response); 
} 
/** 
* send the rak811 to sleep for time  
* specified in millis paramteer 
*/ 
void sleep(unsigned long milliseconds){ 
 sendCommand("at+sleep\r\n"); 
 delay(milliseconds); 
 //send any charcater to wakeup; 
 sendCommand("***\r\n"); 
} 
/** 
* reset board after the specified time delay millisenconds 
* <mode> = 0 Reset and restart module 
= 1 Reset LoRaWAN or LoraP2P stack and Module will reload LoRa 
configuration from EEPROM 
*/ 
void resetChip(int mode, unsigned long delaySec=0){ 
 delay(delaySec); 
 String command = (String)"at+reset=" + mode + (String)"\r\n"; 
 sendCommand(command); 
} 
/** 
* Reload the default parameters of LoraWAN or LoraP2P setting 
*/ 
void reload(unsigned long delaySec){ 
 delay(delaySec); 
 sendCommand("at+reload\r\n"); 
} 
/** 
* Function to set module mode 
* <mode> = 0 LoraWAN Mode (default mode) 
= 1 LoraP2P Mode 
*/ 
void setMode(int mode){ 
 String command = (String)"at+mode=" + mode + (String)"\r\n"; 
 sendCommand(command);   
} 
/** 
* Function to send data to a lora gateway; 
* <type> = 0 send unconfirmed packets 
= 1 send confirmed packets 
<port> = 1-223 port number from 1 to 223 
<data>= <hex value> hex value(no space). The Maximum length of <data> 64 bytes 
*/ 
void sendData(int type, int port, String data){ 
 String command = (String)"at+send=" + type + "," + port + "," + data + (String)"\r\n"; 
 sendCommand(command); 
} 
/** 
* Function to send a join request 
* allowed methods "otaa" and "abp" 
*/ 
void sendJoinReq(String method){ 
 String command = (String)"at+join=" + method + "\r\n"; 
 sendCommand(command); 
} 
/** 
* Function to set the connection config 
* < dev_addr >:<address> 
<address>-------------------4 bytes hex number representing the 
device address from 00000001 – 
FFFFFFFE 
<dev_eui>:<eui> 
<eui>-------------------------- 8-byte hexadecimal number 
representing the device EUI 
<app_eui>:<eui> 
<eui>----------------------------8-byte hexadecimal number 
representing the application EUI 
<app_key>:<key> 
- 11 - 
<key>----------------------------16-byte hexadecimal number 
representing the application key 
<nwks_key>:<key> 
<key>-------------------------16-byte hexadecimal number 
representing the network session key 
<apps_key>:<key> 
<key>------------------------ 16-byte hexadecimal number representing 
the application session key 
<tx_power>:<dbm> 
<dbm>------------------- LoRaWAN Tx Power 
<adr>:<status> 
<status>----------------------------- string value representing 
the state, either “on” or “off”. 
<dr>:<data rate> 
<data rate>-----------------------decimal number representing the 
data rate, from 0 and 4, but within 
the limits of the data rate range for 
the defined channels. 
< public_net >:<status> 
<status>------------------- string value representing 
the state, either “on” or “off”. 
< rx_delay1 >:<delay> 
<delay>-------------------decimal number representing 
the delay between the transmission 
and the first Reception window 
in milliseconds, from 0 to 65535. 
*/ 
void setConnConfig(String key, String value){ 
 sendCommand("at+set_config=" + key + ":" + value + "\r\n"); 
} 

To keep the tutorial short the details of the above code is explained in the tutorial:

https://www.hackster.io/naresh-krish/using-the-rak811-lora-module-with-arduino-a38de8

You can proceed to the AWS IoT Things console to and then Proceed to the Thing that you created for the LoRa node. Once inside the things registry console. Click on the Shadow option in the left Pane. This should show the new value that you have send via the rak811 node.

Congrats!!! you have successfully connected the LoraWAN node readings to your Amazon AWS IoT things attributes.