Software apps and online services
Have you always wanted a dog? A dog you can hug? A dog who smiles at you when you say his name or pet him? But you have no time to walk every day and always clean the whole apartment. Do you want a dog that does not lose its hair and still fills your heart?
We developed Pepper: He listens to you and smiles when you pet him. He can get up and sit down and sometimes he is tired as puppies sometimes are. We are happy to share our project with you. Say hi to pepper from us!
Hardware and software components
Things used in this project
· Lego Mindstorm Ev3 Core Set 45544 - new (Amazon link)
· Echo Dot (Amazon link)
· Micro SD Card 8-32 gb (Amazon link)
· WiFi USB adapter (Amazon link)
· A method to flash the SD card (e.g. balenaEtcher for Windows 10)
· Computer for installing software and programming your EV3 Brick (e.g. Windows 10 Laptop)
The following links provided the basis for the instructions of the Puppy Pepper project:
· Setup (ev3dev, Visual Studio Code): https://www.hackster.io/alexagadgets/lego-mindstorms-voice-challenge-setup-17300f
· Mission 1 (Alexa Gadget, Connect Gadget with Echo Dot): https://www.hackster.io/alexagadgets/lego-mindstorms-voice-challenge-mission-1-f31925
· Mission 3 (Basic functionality): https://www.hackster.io/alexagadgets/lego-mindstorms-voice-challenge-mission-3-4ed812
· PDF instructions to build the puppy with Lego bricks from Lego Education:
1) Set up your EV3 development environment
Before you can get started with the missions, you’ll need to get your EV3 development environment set up. The following instructions will guide you through installing the following:
· ev3dev Software: ev3dev is a Debian Linux-based operating system that runs on several LEGO MINDSTORMS compatible platforms including the LEGO MINDSTORMS EV3. ev3dev gives you the power to use Python 3 to control sensors, motors and pretty much everything else. Learn more about ev3dev.
Follow the getting started instructions on the ev3dev website, which will walk you through:
- Downloading the ev3dev software to your computer
- Flashing your SD Card with the software
- Inserting the SD Card into your EV3 Brick
- Booting up your EV3 Brick
- Getting your EV3 Brick connected to the network via Wifi, or the USB cable that came with LEGO MINDSTORMS EV3
2) Build your Puppy with LEGO bricks
Follow the PDF instructions to build the Puppy with Lego bricks from Lego Education:
Be creative with ears, tail or shape in general - you now have the functional shape that is necessary
3) Import the logic and flash the code to Pepper
Get you code from the attached Zip-File and import into Visual Studio Code:
· Download and unzip alexa-gadgets-mindstorms-PepperProject.zip.
· Select File > “Open folder” from the top menu in Visual Studio Code
· Open the folder ‘alexa-gadgets-mindstorms/folder-puppy-pepper’ you downloaded.
· If necessary: Install extensions as displayed at the bottom of the screen with a callout message.
· If necessary: Click Show Recommendations. If extensions are not recommended, you will need to search manually for ‘ev3dev-browser’
· Once the extensions have installed, click on the Explorer icon on the left of VS Code where you’re able to see your project files.
· Click the arrow next to the EV3DEV DEVICE BROWSER to expand more options.
· Ensure that your EV3 Brick is turned on and has a connection to your computer (USB or Wi-Fi will work), and that your EV3 Brick has the ability to connect to the internet. You should see an IP address in the upper-left of the EV3 Brick interface.
· Click "Click here to connect to a device".
· A box will pop up that lists discovered devices. Select your device, or if you don't see your device, click "I don't see my device...“.
· If you selected I don’t see my device... enter the name or IP-address you want to give your EV3 Brick, and press the Enter key.
· If you click the arrow next to the green dot, you will see information about your EV3 Brick, including battery level and files. When you hover over the EV3DEV DEVICE BROWSER title bar, you will see buttons for Refresh and Send workspace to device.
· Copy the folder-puppy-pepper folder within the alexa-gadgets-mindstorms folder over to your EV3 Brick by clicking the Send workspace to device button. In the bottom-right of VS Code, you will see the status of the files being copied.
4) Register an Alexa Gadget in the Alexa Developer Console
In order for your EV3 Brick to work with Echo Dot device as an Alexa Gadget, you will need to register your gadget in the Alexa Developer Console. Registering your gadget ties your unique gadget to your account, and is required in order for your creation to work properly with Alexa.
· Sign in to https://developer.amazon.com. Sign up or sign in your developer account.
· Click on Developer Console in the upper-right of the page header.
· In the top header, hover over Alexa, and click on Alexa Voice Service.
· Click on Products. You may need to click AVS Program Requirements or Getting Started before you’re able to access the Products option.
· Click on Create Product in the upper-right of the page.
· Fill out the requested information:
- Name: MINDSTORMS EV3
- Product ID: EV3_01
- Product type: Alexa Gadget
- Product category: Animatronic or Figure
- Product description: Whatever you like
- Skip upload image
- Do you intend to distribute this product commercially?: No
- Is this a children’s product or directed to children younger than 13?: No
- Accept the services agreement and click Finish.
· You will see a confirmation popup. Click OK.
· You will be taken to a list of your products. Click on the gadget you just created.
· At the top, you'll see an Amazon ID and Alexa Gadget Secret you'll need for the following steps to create your Alexa Gadget. Keep this ID and Secret private as it uniquely identifies your gadget.
You've now successfully registered your Alexa Gadget!
5) Prepare your EV3 Brick and make sure Bluetooth Power in ON (do not connect manually)
· Your EV3 Brick will be connecting to your Echo device using Bluetooth, which you'll need to enable on your Brick in order for the sample code to work correctly:
- Using the buttons on your EV3 Brick, navigate to Wireless and Networks.
- Select Bluetooth.
- Toggle Powered to ON.
· You should see a Bluetooth icon appear in the header of the interface right next to the Wifi icon.
· At this point, you can switch over to the Visual Code Workspace you created during the setup instructions. The Explorer panel should display the mission folders that are on your computer.
6) Registration and Connect via Bluetooth
· Within the folder-puppy-pepper/folder-puppy-pepper folder, you will see an INI file, and a Python file. Open up the mission-puppy-pepper.ini file, which includes the following configuration details:
amazonId = YOUR_GADGET_AMAZON_ID
alexaGadgetSecret = YOUR_GADGET_SECRET
Alexa.Gadget.StateListener = 1.0 - wakeword
· Replace the YOUR_GADGET_AMAZON_ID and YOUT_GADGET_SECRET with the Amazon ID and Alexa Gadget Secret fro Step (4) above.
· Sync your folder-puppy-pepper folder in your computer workspace to your EV3 Brick. Click on the Send workspace to device button next to the EV3DEV DEVICE BROWSER text that shows up when you hover over the text. When you click this button, you will see the status of the files being copied in the bottom-right of VS Code.
· When you click this button, you will see the status of the files being copied in the bottom-right of VS Code.
· Once the files have copied over to your EV3 Brick, you can run the Python code by navigating to the Python file in the EV3 DEVICE BROWSER, right-click on the Python file, and select Run.
· Note: Once the code has been copied to your EV3 Brick, and run once from your computer, you can also run it using the Brick’s interface - without having to use your computer! In the Brick’s interface select File Browser > alexa-gadgets-mindstorms/folder-puppy-pepper > mission-puppy-pepper.py*. You should see an asterisk next to the file name, which means you can run it directly from the EV3 Brick interface.
· Bluetooth Pairing: Once the program starts, you should hear your EV3 Brick play some tones, and see a prompt in the debug console telling you the friendly name of your gadget, which will look like Gadget000. This means your gadget is in pairing mode.
· You're now ready to pair your EV3 Brick to your Echo device. This pairing process will depend on whether or not your Echo device has a screen. As the Echo Dot does not have a screen you need to pair the device with an App on your mobile phone (see also these instructions for how to pair):
To pair in the Alexa app:
- Select the Devices icon
- Select the Echo device you want to pair to your Alexa Gadget.
- Select Pair Alexa Gadget.
- Select the Alexa Gadget you want to pair.
· Once your device is paired, you should see an acknowledgement that your gadget has paired in the debug console.
· You will also see a message on your EV3 Brick's screen.
7) Create a new Alexa Skill
· Overview to create a new Alexa Skill:
- Skill name MINDSTORMS (NOT: Pepper)
- Language English (US)
- Choose a model Custom
- Choose a method Alexa-Hosted (Node.js)
Remark: Gadget code is written in Python 3. The code for your skill is written in Node.js in order to take advantage of the Alexa-hosted option: https://developer.amazon.com/alexa/console/ask
· In the upper-right of the screen, click on Create Skill.
· Enable the Custom Interface Controller: The Custom Interface Controller allows you to send custom commands to your EV3Brick (Custom Directives), and receive events from your EV3 Brick (Custom Events). For this mission, you will need to enable the Custom Interface Controller, so you can use Custom Directives:
- Click on Interfaces in the left navigation of the Alexa Developer Console.
- Toggle Custom Interface Controller to ON.
- Scroll to the top of the page and click Save Interfaces
· Define the Skill Interaction Model: The Skill Interaction Model defines how you can speak to your skill, and what kind of commands it can expect to respond to. The interaction model includes intents, slots, sample utterances that you define, and program against in your skill’s code. Learn more about the Skill Interaction Model.
- In the Alexa Developer Console, under Interaction Model, click on JSON Editor.
- In the / folder-puppy-pepper folder, you will see a folder called skill-nodejs. Within that folder, there is a model.json file. Copy the interaction model JSON from that file, and paste it into the editor, or drag and drop the JSON file onto the drop zone to upload it.
- After pasting the JSON into the Alexa skill JSON Editor, click Save Model, and then Build Model presented at the top of the console interface. It may take some time for the model to build, so be patient. In the meantime, let’s explore the Intents and Slots in the JSON file.
- An intent represents an action that fulfills a user's spoken request. Intents can optionally have arguments called slots. The combination of the intent and slots will allow to write code that reacts to
· Implement the Skill Logic: With an Alexa-Hosted skill, you can start writing code for your skill directly in the Alexa Developer Console:
- Click on Code in the top navigation bar of the Alexa Developer Console.
- In VS Code, open the index.js file in the folder-puppy-pepper/skill-nodejs/lambda folder.
- Copy the code in the index.js file into the index.js file in the Alexa Developer Console Code Editor.
- Copy the contents of the package.json and util.js files to the respective files in the Alexa Skill Code Editor.
- Create a new file by clicking the New File icon in the upper-left of the Code Editor, and fill in the path and file name as /lambda/common.js
- With the common.js file created, make sure the file is open, and then copy the code in the common.js file from the folder-puppy-pepper/skill-nodejs/ folder in VS Code to the common.js file in the Alexa Skill Code Editor.
· Click Deploy in the upper-right of the Alexa Skill Code Editor. Wait for the deployment process to complete.
· In the top navigation bar of the Alexa Developer Console, click on Test.
· Switch the testing from Off to Development using the dropdown under the navigation bar.
· In the panel on the left, you can either type “open pepper”, or say that utterance when you press the microphone button (you will have opted to enable the microphone on your computer for this to work).
· You should get a response along the lines of, “I couldn't find an EV3 Brick connected to this Echo device. Please check to make sure your EV3 Brick is connected, and try again”, which you specified in the LaunchRequestHandler.
Your skill is working! Let’s complete the other side of the experience by reviewing and running the EV3 code to get EV3RSTORM to react to your skill.
8) Test EV3RSTORM with the Alexa skill
· You will need to run the Python code before starting the skill!
· Now comes the fun part! Make some room for EV3RSTORM, and try the following commands to see how EV3RSTORM reacts (the logs will be written to the OUTPUT in the Visual Studio Code console):
- “Alexa, open mindstorms”
… and then:
- «pepper bark»
- «pepper sit down»
- «pepper stand up»
- «pepper pee» / «pepper look there is a tree» / «pepper you can pee»
- «pepper sit down»
- «pepper stretch»
… and then:
- Pet pepper on his back and look at his face
· Note: You can always stop the program by clicking the Stop button at the top of the screen in Debug mode, or pressing the back button on your EV3 Brick.