Alexa / Node RED / Zentri Mashup

Application Description

The main function of the application is to bridge Amazon’s Alexa voice interface to IBM’s cloud service – Bluemix, to the Avnet BCM4343W IoT Starter Kit board, highlighting this application's effectiveness in smart home design. This is done through an HTTPS connection between Amazon and IBM and an MQTT connection between IBM and the Avnet board.

In the first HTTPS connection, information travels from Alexa to IBM in the form of “intents”, which Alexa decodes from the speech recordings. The Bluemix code then deciphers these intents to extract necessary data for publishing to the subscribed devices. Bluemix also simultaneously renders a speech response to the command posed, sent back to Amazon through the same connection.

Between IBM and the Board, an MQTT connection is used for efficiency. Messages communicated through this full duplex channel go both ways. For instance, if asked to publish WiFi information, the board will commence periodic publishing while at the same time still receiving and processing incoming commands.

There are three different types of speech that this application accepts. Firstly, a simple command such as “GO” or “STOP” follows a simple network chain from Alexa to Bluemix to the board, which then modifies the voltage level at the appropriate GPIO. The addition of a relay board extends possibilities by controlling devices that require higher voltage such as motors, actuators etc. A 5V relay Shield was utilized in this example, for reliable operation a 3.3V relay board is recommended.

Secondly, for requests such as WiFi status, a publishing command must first be initiated. The board will then periodically publish info to Bluemix (eg. 1x/second), where basic “QuickStart” charting is provided (visualization also possible via a custom dashboard). This data can also be retrieved via Alexa in the form of a request, which is a command preceded with the word “Get”. Lastly, this application takes in speech regarding the operation of the program such as “Start Avnet”, “Goodbye” or “Help”.

Instructions

0. Assemble your hardware to look like the following images.

• Make sure the 4 connected pins of the 8x8 LED Matrix are lined up on the left side of the Pmod interface (ie. closest to the Micro USB port)

1. Install Alexa to Avnet Firmware

• If not already installed, download and install the latest version of ZentriOS SDK (link posted at end of this tutorial). When finished, open Zentri.

• Download the provided AV04.zip project source files and extract the AV04 folder to your desktop. Copy the “Alexa_to_Avnet” folder in the “AV04 Project Files” folder to the Zentri workspace (...\Zentri\workspace).

• In the Zentri interface, right click in the Zentri Project Explorer, then click “Import…”

• This will open up the Import menu - select “Existing Projects into Workspace” under the General folder, then click next.

• Browse for the Zentri Workspace folder, select the Alexa_to_Avnet folder to import, click “OK” and then “finish”. You should now see an “Alexa_to_Avnet” Project in your project explorer.

• Connect a USB cable from the Avnet BCM4343W IoT Starter Kit to your computer. Select the “ZentriOS” tab on the top bar and click “Setup Device and SDK”. Select the appropriate version when prompted and click “Finish” when complete.

2. Run Alexa to Avnet

• To run the program, right click on the “Alexa_to_Avnet” folder, select “ZentriOS”, then click on

“Build / Download / Run”

• After successful build and download of this application to the hardware, open a serial console application to view the output from the hardware. Make sure that the serial console is properly configured and that the Baud rate is set to 115200.

• You will need to setup a WiFi connection for your board in the serial console. Start by pressing the enter key one time to enter command mode, then type “network_up -s” into the console.

• Locate your network and its ‘#’ on the list that appears. Enter your network number into the console followed by the password. Once connected, type “save” to have the device remember the network.

• Now, press the reset button on the board to reboot the program.

• Copy the DeviceID (highlighted in the figure below) that was reported in the serial console and store it somewhere accessible, as it will be used later on.

3. Set up your Bluemix account:

• Register and log on to the IBM Bluemix site: https://new-console.ng.bluemix.net/

• If this is your first time logging on, you will be prompted with the following dialogue. Make sure the region is selected as “US South” and enter a unique organization name.

• Next, give a unique name for your first space.

• When finished, click “I’m Ready” and give Bluemix a minute to set things up.

4. Create your Bluemix application:

• You should now be redirected to the new Console page of IBM Bluemix: https://new-console.ng.bluemix.net/#overview. Scroll down and click on the “Take Advantage of IoT” box

• Give your application a unique name - we suggest the name “Alexa to Avnet XXXX”, with “XXXX” being the last four digits of your device ID from earlier. Leave the rest of the fields as is, then click “create”.

• Give Bluemix 2-10 minutes, to start your application. You will see the status change to indicate this.

5. Program the Node RED Editor:

• From your dashboard, click on “View App” on the right hand side to access the Node RED editor.

• Click the red “Go to your Node-RED flow editor” button

• You should now see two default “flows”. For this demo, we will import a new flow specific to the application. To do so, first create a new flow by clicking the “+” button to the right of the screen.

• A new tab should appear in your workspace called “Flow 2” - click it to open up the blank new workspace, then navigate through the icon of the three horizontal bars -> Import -> Clipboard and paste in the Node RED code provided. Click anywhere to drop the flow.

• You should now see a flow resembling the following on your screen

Do click on the “Incoming from Alexa” node in the upper left corner of the flow. A window should appear that will allow you to edit the https properties. In this example, we created the URL with /AlexaDemo such that Amazon would send voice information collected from Alexa to the URL https://alexa-to-avnet-XXXX.mybluemix.net/AlexaDemo . Edit this if you wish.

Double click on the node “Output to <Your board name>” on the left side and enter the device ID of your device collected at the end of part 2 of this tutorial. Rename the node to replace “<Your board name>” with the name of your board.

Edit the nodes “Input from <Your board name>” and “Output to dashboard” similarly. When finished, click “Deploy” in the upper right corner. You are now done with the Node RED editor.

6. Define the Alexa Voice Interface:

In order to work with the Alexa Voice Interface, you will first need to create an Amazon account and an Alexa skill through the Alexa Skills Kit (ASK):

https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit

• Click on “Create a skill now” on the right side of the screen

• Click “Add a New Skill” on the upper right hand corner of the screen

• You will then need to give your skill a skill name between 2 - 50 characters long, as well as an invocation name, which is a phrase you would say to Alexa to invoke your skill.

The Intent Schema maps various sayings to specific intents. Copy the Intent Schema located in the “ASK Code” document in the AV04 Project Files (everything between the “Intent Schema” header and the Custom slot Types header) to your skill.

Next you will need to add slot types - click “Add Slot Type” and create a slot for “BCM_IN_CMD_LIST” by entering the values “light sensor” and “wifi”. Press “Save” when finished and continue.

Create the rest of the slots similarly using the ASK code document, until your slots match the following:

Lastly, copy the sample utterances included in the ASK Code file and continue. Choose “HTTPS” for your endpoint and use the Node RED URL in step 5 (This is also in the ASK Code text file - replace the “XXXX” with the four digits of your device ID. Your link should also include edits in the http node from Node RED). Check “No” for Account linking and continue.

Check the middle wildcard option and click “save”. Your Alexa skill is now complete. Feel free to use the next section (test) to try out your voice interface in lieu of an Alexa product.

7. “Start Avnet”.

This command will start a conversation with Alexa. After Alexa’s reply, try saying any of the other commands. Note that the get temperature command is yet to be implemented in the version of the code released here.

To directly command Alexa without the “Start Avnet” phrase in front, simply say “Ask Alexa” followed by the desired command. For example, “Ask Alexa Go” will automatically turn on the green LED.

8. What’s next?

• Try creating a dashboard with IBM Bluemix!

• Play around with the node RED editor to change voice responses from Alexa!

By bridging the Alexa Voice Interface to IBM Bluemix in this mashup, we open up a door to many more possibilities to utilize your voice data. Services such as Node RED allow for visual wiring between hardware, API's and other online services, such as Cloudant DB, Twilio, IFTTT etc. in a much more intuitive way. Take your time to play with Bluemix and see what you can come up with!