Ask The Barn

Voice control adds another dimension to the Dashboard I had created using Freeboard. The project consists of two Raspberry Pi computers , a sensor tag and a webcam with a microphone. One Worker Raspberry Pi will deploy the sensor data to the IBM Blue Mix cloud and using Node Red connect to Alexa. The second Commander Raspberry Pi can be at any location and connects to the Amazon cloud to speak to the IBM Blue Mix cloud and reports the status on indoor weather locations.  The Worker Pi could be at a shelter , barn , greenhouse or any enclosure where knowing temperature, humidity and lux are vital to the animal or plant's wellbeing. The Commander Pi could be in any location. Voice control adds a convenient, inexpensive and fast way to check indoor weather conditions.

Texus Instruments Sensor Tag (CC2650)

The CC2650 includes 10 onboard sensors. The setup is simple. You pull the plastic tab to activate the sensors and download either the free Android or IPhone App to visualize the data. The Tag has a battery life of a year. The data will be sent to Bluetooth on the Pi and then sent to the IBM Bluemix Cloud.

Microphone

I chose the low cost Logitech c170 webcam with a microphone included. When I first tested this with aplay I received just static noise. So I had to configure my microphone for the raspberry pi. First you need to load the drivers. 

sudo modprobe snd_bcm2835 
sudo nano /etc/modprobe.d/alsa-base.cong 
options snd-usb-audio index=1 

To save it you need to CTL+X. If the module is blank just add the code, Same for the next step.

Then set up asound.conf with the following code:

sudo nano /etc/asound.conf  
pcm.!default { 
type plug 
slave { 
pcm "hw:1,0" 
} 
} 
ctl.!default { 
type hw 
card 1 
} 

To save it you need to CTL+X before you reboot.

Run this command and say something:

arecord -D plughw:1,0 -f cd test.wav 

Play it back

aplay test.wav 

I received my recording back..”testing 123” and know my mic is working.

As a side note – I’m pushing the sound to my RCA TV using HDMI connection to the Pi. I had to run a few commands:

sudo nano /boot/config.txt 

Uncomment the line by deleting #

hdmi_drive=2 

So the hardware is working for the Commander Pi. Now we need to install Alexa.

Setting up Alexa on the Commander Pi

STEP 1 VNC SERVER

Install VNC Server

VNC is a graphical desktop sharing system that will allow you to remotely control the desktop interface of your Raspberry Pi from another computer.  You need to install it on the Raspberry Pi and your laptop or computer. We will start with the Raspberry Pi.

sudo apt-get install tightvncserver 

Start VNC Server

To start VNC Server, type

tightvncserver 

You'll be asked to set a password to access the Pi. The password is maximum 8 characters.

Installing VNC on my laptop

I’m on a Windows 10 Laptop and now need to install a VNC client. The machine is 64 bit so I will be installing the Windows (64-bit) version.

http://www.tightvnc.com

During installation you'll be offered the choice of Typical, Custom or Complete. You only need the VNC client and not the server, so choose Custom. Then select TightVNC Server and choose Entire feature will be unavailable. Click Next. Uncheck the option about Windows Firewall and click Next again, then Install.

Once the installation is complete you should find TightVNC Viewer under the start menu. When you run it you'll be presented with the dialog box. You will need to enter the IP address of the Raspberry Pi followed by the screen number (:0 or :1). For example: 192.168.0.6:1

Click on connect and you should see your raspberry pi desktop

STEP 2 : VLC

Install VLC

Get VLC media player by typing:

sudo apt-get install vlc-nox vlc-data 

Test that VLC is installed correctly. Type the following into the terminal:

whereis vlc 

This will provide you with the installation path.

vlc: /usr/bin/vlc /usr/lib/vlc /usr/share/vlc /usr/share/man/man1/vlc.1.gz 

Set the environment variables for VLC . Type the following into the terminal:

export LD_LIBRARY_PATH=/usr/lib/vlc   
export VLC_PLUGIN_PATH=/usr/lib/vlc/plugins 

Check if the environment variables were set successfully:

echo $LD_LIBRARY_PATH 
> /usr/lib/vlc 
echo $VLC_PLUGIN_PATH 
> /usr/lib/vlc/plugins 

STEP 3 Apply for a free Amazon Developer Account

Get a free Amazon developer account if you do not already have one.

STEP 4: Download the sample app and dependencies on your Raspberry Pi

Download the sample app zip file from the Github repo. Please note that by downloading this package, you are agreeing to the Alexa Voice Service Agreement.

https://github.com/amzn/alexa-avs-raspberry-pi

Copy and expand the .zip file on your Raspberry Pi

Unless you downloaded the zip directly onto your Pi, copy and then expand the zip file on your Pi to/home/pi/Desktop/alexa-avs-raspberry-pi-master/. I downloaded from my laptop and moved the file to the Pi using FileZilla.

Step 5: Installing dependencies - Node, JDK, Maven

sudo apt-get update&&sudo apt-get upgrade 

Set up the apt-get repo source:

curl -sL https://deb.nodesource.com/setup | sudo bash - 

Install Node:

sudo apt-get install nodejs 

Install NPM:

Change directories to /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/companionService

Install the dependencies:

npm install 

Install Java Development Kit

You need to have Java Development Kit (JDK) version 8 or higher installed on your Pi. To make things easier, we've included a script that will install the latest version of JDK on your Pi and remove older JDK versions.

cd /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/javaclient 

Run the installation script:

./install-java8.sh 

You will get a message from Oracle Java installer that you must accept the Terms of Service for Java SE Platform, press Enter. Then, press Tab, and then Enter to say “Yes” to the Terms of Service.

Verify your version of JDK

Now verify the commands with the -version option:

java -version 

>> java version "1.8.0_91"

>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)

>> Java HotSpot(TM) Client VM (build 25.91-b14, mixed mode)

javac -version 

>> javac 1.8.0_91

Make sure your JDK and ALPN versions match

Using the File Manager on the Pi, navigate to the following directory:

/home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/javaclient 

In a text editor, open pom.xml and locate <alpn-boot.version>xxx</alpn-boot.version>. Confirm the ALPN version matches your JDK version using the table located at eclipse.org/jetty. If the versions match no further action is required. If the versions do not match, update the pom.xml file with the correct ALPN version and save. My version was fine.

Install Maven

Download Maven

Download the Binary tar.gz file apache-maven-3.3.9-bin.tar.gz from maven.apache.org Again I used FileZilla to upload the file to the Raspberry Pi.

https://maven.apache.org/

Extract the contents

Go to:

cd /home/pi/Downloads 

Extract the contents of the tarball to the /opt directory

sudo tar xvf apache-maven-3.3.9-bin.tar.gz  -C /opt 

Tell your shell where to find maven. You’ll do this in the system profile settings so it is available to all users. Create a new file /etc/profile.d/maven.sh, by typing the following in the terminal:

sudo touch /etc/profile.d/maven.sh   
sudo nano /etc/profile.d/maven.sh 

In the editor, add the following to maven.sh:

export M2_HOME=/opt/apache-maven-3.3.9   
export PATH=$PATH:$M2_HOME/bin 

Type ctrl-X and then Y, and then press Enter to save the changes to the file.

Log out and back into the Pi so the profile script takes effect. You can test that it is working with the following command:

mvn -version 

>> Apache Maven 3.3.9

STEP 6: Generate self-signed certificates

Install SSL

Get openssl:

sudo apt-get install openssl 

Verify install

whereis openssl 

> openssl: /usr/bin/openssl /usr/share/man/man1/openssl.lssl.gz

Change directories to

cd /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/javaclient 

Run the certificate generation script:

./generate.sh 

You will be prompted for some information:

These must be entered exactly as they appear below. It will not work if you change the entries !

• When prompted for a product ID, enter my_device

• When prompted for a serial number (DSN), enter 123456

• When prompted for a password, press enter to leave it blank.

A private key would be generated for you.

STEP 6: Getting started with Alexa Voice Service

Register your product and create a security profile.

• Login to Amazon Developer Portal - developer.amazon.com

• Click on Apps & Services tab -> Alexa -> Alexa Voice Service -> Get Started

• In the Register a Product Type menu, select Device.

• Fill in and save the following values:

Device Type Info

• Device Type ID: my_device

• Display Name: My Device

• Click Next

Security Profile

• Click on the Security Profile drop down and choose “Create a new profile”

General Tab

• Security Profile Name: Alexa Voice Service Sample App Security Profile

• Security Profile Description: Alexa Voice Service Sample App Security Profile Description

Click Next. Client ID and Client Secret will be generated for you.

1. Now click on the Web Settings Tab 1.Make sure the security profile you just created is selected in the drop-down menu, then click the "Edit" button.

2. Allowed Origins: Click "Add Another" and then enter https://localhost:3000 in the text field that appears.

3. Allowed Return URLs: Click "Add Another" and then enter https://localhost:3000/authresponse in the text field that appears.

4. Click Next

Device Details

1. Image: Save the following test image to your computer, then upload it:

2. Category: Other

3. Description: Alexa Voice Service sample app test

4. What is your expected timeline for commercialization?: Longer than 4 months / TBD

5. How many devices are you planning to commercialize?: 0

6.Click Next

You are now ready to generate self-signed certificates.

Enable Security Profile

1. Open a web browser, and visit https://developer.amazon.com/lwa/sp/overview.html

2. Near the top of the page, select the security profile you created earlier from the drop down menu and click Confirm.

3. Enter a privacy policy URL beginning with http:// or https://. For this example, you can enter a fake URL such as http://example.com.

4. [Optional] You may upload an image as well. The image will be shown on the Login with Amazon consent page to give your users context.

5. Click Save.

Next to the Alexa Voice Service Sample App Security Profile, click Show Client ID and Client Secret. This will display your client ID and client secret. Save these values. You’ll need these. Save to NotePad !

STEP 7 Update the config files

Login to the Raspberry Pi via VNC

Open config.js:

nano /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/companionService/config.js 

Edit the following values:

• clientId: Paste in the client ID that you noted in the previous step as a string.

• clientSecret: Paste in the client secret that you noted in the previous step as a string.

While pasting the values, be careful to enclose them in single quotes.

Type ctrl-X and then Y, and then press Enter to save the changes to the file.

STEP 8 Run the server

IMPORTANT: This must be done via VNC, not SSH. So, make sure you’re logged into your Pi via VNC.]

Login to the Raspberry Pi via VNC In your terminal window drill down to:

cd /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/companionService     
npm start 

The server is now running on port 3000 and you are ready to start the client.

STEP 9 Start the client

Open a new terminal window/tab (SHIFT+CTRL+T in Raspbian) and navigate to:

cd /home/pi/Desktop/alexa-avs-raspberry-pi-master/samples/javaclient 

Build the app. Download dependencies and build the app by typing:

mvn install

When the installation is completed, you will see a “Build Success” message in the terminal.

Run the client app:

You are now ready to run the client app by typing:

mvn exec:exec 

Obtain Authorization from Login with Amazon 1.When you run the client, a window should pop up with a message that says something similar to:

Please register your device by visiting the following website on any system and following the instructions: https://localhost:3000/provision/d340f629bd685deeff28a917 Hit OK once completed.

DO NOT CLOSE THIS WINDOW YET

Copy the URL from the popup window and paste it into a web browser. In this example, the URL to copy and paste is https://localhost:3000/provision/d340f629bd685deeff28a917.

• You'll be taken to a Login with Amazon web page. Enter your Amazon credentials.

• You'll be taken to a Dev Authorization page, confirming that you’d like your device to access the Security Profile created earlier.

Click Okay.

• You will now be redirected to a URL beginning with https://localhost:3000/authresponse followed by a query string. The body of the web page will say device tokens ready.

• Return to the Java application and click the OK button. The client is now ready to accept Alexa requests.

• Click the Start Listening button and wait for the audio cue before beginning to speak. It may take a second or two before you hear the audio cue.

Press the Stop Listening button when you are done speaking.

So now Alexa is installed on the Raspberry Pi let's test it out!

Setting up the Sensor Tag on the Worker Raspberry Pi to send Sensor Data to IBM Blue Mix

You need to sign up for a free trial from IBM. You need a Bluetooth dongle and install the software to register the dongle. Make sure you update the firmware.

sudo apt-get update 
sudo apt-get install bluetooth bluez blueman 

After the install you can run lsusb and see your bluetooth in the list:

At your terminal use Git command to clone the repository . Found here - https://github.com/IBM-Bluemix/iot-sensor-tag

git clone https://github.com/IBM-Bluemix/iot-sensor-tag 

Move to the publish directory and execute the command npm install – You may see some warnings but as long as they are not errors it should be okay.

Now lets set up our Raspberry Pi with Bluemix Console for IBM. Click on Catalog.

Select Internet of Things on the left. Now select Internet of Things:

Under Connect you device – click on Launch dashboard:

Select Create Device Type:

Select Create device type:

Name your Device. Add the MAC address for the Pi. The MAC address can be found using ifconfig. All other fields are not mandatory

Now we can start the program from the publish directory. Before we do this we must create a config.properties within the publish directory on our Worker Raspberry Pi:

sudo nano  config.properties

Run:

Node sensor-tag.js  

Make sure your sensor tag is connected and you should see this in Putty.

We now have Alexa working on Commander Pi and the Worker Pi is deploying sensor data to the Bluemix cloud.  Next we will use Node Red to have both Raspberry Pi communicating with each other.

Node Red

This is a powerful tool developed by IBM. The cool thing about Node Red is will create a Url for us to use as a web interface with Alexa.  This also provides an opportunity to develop customized skills set to ask about temperature, humidity and lux within the enclosure. The possibilities are endless to customize this for any project. I have demonstrated some basic Alexa commands but the next step brings the project to a new level.

Step 1: Setting up Skills Set for  Ask The Barn

Once you have Alexa set up on your Raspberry Pi you can customize commands . I have demonstrated some basic commands in the video but the sky is the limit as to what you can do with the skills kit.

Setting up the Skill Set

Log into Bluemix and select FoxTracker

FoxTracker Dash Board includes the url for Routes. We are going to be using this as an endpoint in Alexa. My route is FoxTracker.mybluemix.net. Later I will use this url and the url I set up for the http node (more on that later)

Log into Amazon Developer website and select Alexa Skills Set.

Name your skill and choose a Invocation Name. I simply called mine "the barn". So in this demo I will be saying "Alexa ask the barn what is the temperature ?"

Set up the Intent Schema. This is flexible. I set up the parameters I wish to test.  In my case it will be Humidity, Temperature and Lux.

Set up Utterances. You can provide a wide range of how the user may question Alexa. For example : "What is the temperature", "how is the temperature ?" or "Temperature Please ?"

Setup your Endpoint. Copy your url from Bluemix and add the url you specified in Node Red ( more on this to come)

Select the second option for wildcard certificate.

Test You Utterance ! Make sure your EndPoint is correct.

Bringing it all together with Node Red

This is working with the Worker Pi to set up nodes for Alexa and our Sensor Tag:

The first node will the http in node. I  have named this /fox so the Endpoint in Alexa  Skills was my app route appended with /fox. The Endpoint is https://foxtracker.mybluemix/fox.

Now we set up a switch for Request type. I believe this is standard and does not change.

Set up another Switch node for the Intents you set up on the Alexa Skills web site:

Set up a Template node for Alexa response:

Add json node  and http out node:

Done with the Alexa connection ! Now we want to trigger the temperature data . We need to add the IBM iot node:

Now extract the temperature data:

Summary

This is my initial build and I still have some work to do. The following is a sample asking Alexa what is the temperature in the barn: