Siddharth Agarwala
Created December 18, 2019

Alexa Powered LEGO Mindstorms Pet Carer

In this project, I have created a LEGO Mindstorms robot that works with Alexa and EV3Dev which takes care of your pet.

IntermediateFull instructions provided47

Things used in this project

Story

Read more

Custom parts and enclosures

Food giver

You can deconstruct this studio 2.0 file and see how to build the piece.

Treat Giver

You can deconstruct this studio 2.0 file and see how to build the piece.

Schematics

Build Instructions

Look at the cad files to build everything and then put it together using this guide.

Code

Python code

Python
Use this in Visual Studio Code with the EV3Dev Browser extension and Python.
#!/usr/bin/env python3

##..............................................................................................
##                                      Imports
##..............................................................................................

import os
import sys
import time
import logging
import json
import random
import threading

from enum import Enum
from agt import AlexaGadget

from ev3dev2.led import Leds
from ev3dev2.sound import Sound
from ev3dev2.motor import OUTPUT_D, OUTPUT_B, OUTPUT_C, SpeedPercent, LargeMotor, SpeedRPM
from ev3dev2.sensor.lego import ColorSensor, TouchSensor

##..............................................................................................
##                                       Logging
##..............................................................................................

# Set the logging level to INFO to see messages from AlexaGadget
logging.basicConfig(level=logging.INFO, stream=sys.stdout, format='%(message)s')
logging.getLogger().addHandler(logging.StreamHandler(sys.stderr))
logger = logging.getLogger(__name__)

##..............................................................................................
##                                  Command definitions
##..............................................................................................

class Command(Enum):
    """
    The list of preset commands and their invocation variation.
    These variations correspond to the skill slot values.
    """
    treat = ['treat']
    dinner = ['dinner']
    Dinnerend = ['end dinner']
    introduce = ['introduce']
##..............................................................................................
##                                      setup
##..............................................................................................


class MindstormsGadget(AlexaGadget):
    """
    A Mindstorms gadget that performs movement based on voice commands.
    three commands are available, Treat, Dinner, End Dinner, and Introduce.
    """

    def __init__(self):
        """
        Performs Alexa Gadget initialization routines and ev3dev resource allocation.
        """
        super().__init__()

        # Ev3dev initialization
        self.leds = Leds()
        self.sound = Sound()
        self.Foodgiver = LargeMotor(OUTPUT_B)
        self.Treatgiver = LargeMotor(OUTPUT_C)
        self.color = ColorSensor()
        self.touch = TouchSensor()
        
##.................................................................................................
##                                      Pairing
##.................................................................................................

    def on_connected(self, device_addr):
        """
        Gadget connected to the paired Echo device.
        :param device_addr: the address of the device we connected to
        """
        self.leds.set_color("LEFT", "GREEN")
        self.leds.set_color("RIGHT", "GREEN")
        logger.info("{} connected to Echo device".format(self.friendly_name))

    def on_disconnected(self, device_addr):
        """
        Gadget disconnected from the paired Echo device.
        :param device_addr: the address of the device we disconnected from
        """
        self.leds.set_color("LEFT", "BLACK")
        self.leds.set_color("RIGHT", "BLACK")
        logger.info("{} disconnected from Echo device".format(self.friendly_name))

##.................................................................................................
##                              Mindstorms Gadget controll
##.................................................................................................

    def on_custom_mindstorms_gadget_control(self, directive):
        """
        Handles the Custom.Mindstorms.Gadget control directive.
        :param directive: the custom directive with the matching namespace and name
        """
        try:
            payload = json.loads(directive.payload.decode("utf-8"))
            print("Control payload: {}".format(payload), file=sys.stderr)
            control_type = payload["type"]
            if control_type == "command":
                # Expected params: [command]
                self._activate(payload["command"])
                

        except KeyError:
            print("Missing expected parameters: {}".format(directive), file=sys.stderr)

    ##.................................................................................................
    ##                            My code for the Pet Nanny
    ##.................................................................................................
    
    

    def _activate(self, command):
        """
        Handles preset commands.
        :param command: the preset command
        """
        
        print("Activate command: ({})".format(command), file=sys.stderr)

        #Introduce
        if command in Command.introduce.value:
            gadget.sound.speak('Hello. I am your personal pet Nanny.')
            gadget.sound.speak('How may I help you?')
        #Give a Treat
        if command in Command.treat.value:      
            while self.color.reflected_light_intensity < 8:
                self.Treatgiver.on(SpeedPercent(-60))
            self.Treatgiver.off()
            self.Treatgiver.on_for_rotations(SpeedPercent(-100), 100/360)

        #Push out dinner
        if command in Command.dinner.value:
            self.Foodgiver.on_for_rotations(SpeedPercent(100), -983/360)

        #Pull back dinner
        if command in Command.Dinnerend.value:
            self.Foodgiver.on_for_rotations(SpeedPercent(100), 983/360)


    

##.................................................................................................
##                                      End scripts
##.................................................................................................


if __name__ == '__main__':

    gadget = MindstormsGadget()

    # Set LCD font and turn off blinking LEDs
    os.system('setfont Lat7-Terminus12x6')
    gadget.leds.set_color("LEFT", "BLACK")
    gadget.leds.set_color("RIGHT", "BLACK")

    # Startup sequence
    gadget.sound.play_song((('C4', 'e'), ('D4', 'e'), ('E5', 'q')))
    gadget.leds.set_color("LEFT", "GREEN")
    gadget.leds.set_color("RIGHT", "GREEN")

    # Gadget main entry point
    gadget.main()

    # Shutdown sequence
    gadget.sound.play_song((('E5', 'e'), ('C4', 'e')))
    gadget.leds.set_color("LEFT", "BLACK")
    gadget.leds.set_color("RIGHT", "BLACK")

Model code

JSON
Use this in the model portion of the Alexa developer console
{
  "interactionModel": {
      "languageModel": {
          "invocationName": "pet.nanny",
          "intents": [
              {
                  "name": "AMAZON.CancelIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.HelpIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.StopIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.NavigateHomeIntent",
                  "samples": []
              },
              {
                  "name": "SetCommandIntent",
                  "slots": [
                      {
                          "name": "Command",
                          "type": "CommandType"
                      }
                  ],
                  "samples": [
                      "give {Command}",
                      "give a {Command}",
                      "activate {Command}",
                      "{Command}"
                  ]
              }
          ],
          "types": [
              {
                  "name": "CommandType",
                  "values": [
                      {
                          "name": {
                              "value": "treat"
                          }
                      },
                      {
                          "name": {
                              "value": "dinner"
                          }
                      },
                      {
                          "name": {
                              "value": "end dinner"
                          }
                      },
                      {
                        "name": {
                            "value": "introduction"
                        }
                      }
                  ]
              }
          ]
      }
  }
}

Index

JavaScript
The code portion of the Alexa Developer console. Replace the Index file.
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

// This sample demonstrates sending directives to an Echo connected gadget from an Alexa skill
// using the Alexa Skills Kit SDK (v2). Please visit https://alexa.design/cookbook for additional
// examples on implementing slots, dialog management, session persistence, api calls, and more.

const Alexa = require('ask-sdk-core');
const Util = require('./util');
const Common = require('./common');

// The namespace of the custom directive to be sent by this skill
const NAMESPACE = 'Custom.Mindstorms.Gadget';

// The name of the custom directive to be sent this skill
const NAME_CONTROL = 'control';

const LaunchRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'LaunchRequest';
    },
    handle: async function(handlerInput) {

        let request = handlerInput.requestEnvelope;
        let { apiEndpoint, apiAccessToken } = request.context.System;
        let apiResponse = await Util.getConnectedEndpoints(apiEndpoint, apiAccessToken);
        if ((apiResponse.endpoints || []).length === 0) {
            return handlerInput.responseBuilder
            .speak(`I couldn't find an EV3 Brick connected to this Echo device. Please check to make sure your EV3 Brick is connected, and try again.`)
            .getResponse();
        }

        // Store the gadget endpointId to be used in this skill session
        let endpointId = apiResponse.endpoints[0].endpointId || [];
        Util.putSessionAttribute(handlerInput, 'endpointId', endpointId);

        return handlerInput.responseBuilder
            .speak("Welcome, you may now start issuing pet nanny commands")
            .reprompt('Try saying Dinner to give dinner, or Treat to give a treat, or introduce to introduce itself')
            .getResponse();
    }
};

// Construct and send a custom directive to the connected gadget with data from
// the SetCommandIntent request.
const SetCommandIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetCommandIntent';
    },
    handle: function (handlerInput) {

        let command = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Command');
        if (!command) {
            return handlerInput.responseBuilder
                .speak("Can you repeat that?")
                .reprompt("What was that again?").getResponse();
        }

        const attributesManager = handlerInput.attributesManager;
        let endpointId = attributesManager.getSessionAttributes().endpointId || [];

        // Construct the directive with the payload containing the parameters
        let directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'command',
                command: command,
                
            });

        return handlerInput.responseBuilder
            .speak(`Snowy ${command}`)                         //Change snowy to the name of your dog.
            .reprompt('Try saying Dinner to give dinner, or Treat to give a treat, or introduce to introduce itself')
            .addDirective(directive)
            .getResponse();
    }
};

// The SkillBuilder acts as the entry point for your skill, routing all request and response
// payloads to the handlers above. Make sure any new handlers or interceptors you've
// defined are included below. The order matters - they're processed top to bottom.
exports.handler = Alexa.SkillBuilders.custom()
    .addRequestHandlers(
        LaunchRequestHandler,
        SetCommandIntentHandler,
        Common.HelpIntentHandler,
        Common.CancelAndStopIntentHandler,
        Common.SessionEndedRequestHandler,
        Common.IntentReflectorHandler, // make sure IntentReflectorHandler is last so it doesn't override your custom intent handlers
    )
    .addRequestInterceptors(Common.RequestInterceptor)
    .addErrorHandlers(
        Common.ErrorHandler,
    )
    .lambda();

Common

JavaScript
Use in the code section of the Alexa Developer console. Replace the Common.js file with this.
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict'

const Alexa = require('ask-sdk-core');

const HelpIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.HelpIntent';
    },
    handle(handlerInput) {
        const speakOutput = 'Try saying Dinner to give dinner, or Treat to give a treat, or introduce to introduce itself';

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};
const CancelAndStopIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && (Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.CancelIntent'
                || Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.StopIntent');
    },
    handle(handlerInput) {
        const speakOutput = 'Goodbye!';
        return handlerInput.responseBuilder
            .speak(speakOutput)
            .getResponse();
    }
};
const SessionEndedRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'SessionEndedRequest';
    },
    handle(handlerInput) {
        // Any cleanup logic goes here.
        return handlerInput.responseBuilder.getResponse();
    }
};

// The intent reflector is used for interaction model testing and debugging.
// It will simply repeat the intent the user said. You can create custom handlers
// for your intents by defining them above, then also adding them to the request
// handler chain below.
const IntentReflectorHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest';
    },
    handle(handlerInput) {
        const intentName = Alexa.getIntentName(handlerInput.requestEnvelope);
        const speakOutput = `${intentName}`; 

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt("I don't understand this command, try again")
            .getResponse();
    }
};

// Generic error handling to capture any syntax or routing errors. If you receive an error
// stating the request handler chain is not found, you have not implemented a handler for
// the intent being invoked or included it in the skill builder below.
const ErrorHandler = {
    canHandle() {
        return true;
    },
    handle(handlerInput, error) {
        console.log(`~~~~ Error handled: ${error.stack}`);
        const speakOutput = `Sorry, I had trouble doing what you asked. Please try again.`;

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};

// The request interceptor is used for request handling testing and debugging.
// It will simply log the request in raw json format before any processing is performed.
const RequestInterceptor = {
    process(handlerInput) {
        let { attributesManager, requestEnvelope } = handlerInput;
        let sessionAttributes = attributesManager.getSessionAttributes();

        // Log the request for debug purposes.
        console.log(`=====Request==${JSON.stringify(requestEnvelope)}`);
        console.log(`=========SessionAttributes==${JSON.stringify(sessionAttributes, null, 2)}`);
    }
};

module.exports = {
    HelpIntentHandler,
    CancelAndStopIntentHandler,
    SessionEndedRequestHandler,
    IntentReflectorHandler,
    ErrorHandler,
    RequestInterceptor
    };

Package

JSON
Replace the Package.JSON file in the Alexa developer console with this.
{
  "name": "agt-mindstorms",
  "version": "1.1.0",
  "description": "A skill using AGT with Lego Mindstorms that takes care of your pet.",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Amazon Alexa",
  "license": "ISC",
  "dependencies": {
    "ask-sdk-core": "^2.6.0",
    "ask-sdk-model": "^1.18.0",
    "aws-sdk": "^2.326.0",
    "request": "^2.81.0"
  }
}

Util

JavaScript
Replace the Util.js file in the Alexa Developer Console with this.
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict';

const Https = require('https');

/**
 * Build a custom directive payload to the gadget with the specified endpointId
 * @param {string} endpointId - the gadget endpoint Id
 * @param {string} namespace - the namespace of the skill
 * @param {string} name - the name of the skill within the scope of this namespace
 * @param {object} payload - the payload data
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#respond}
 */
exports.build = function (endpointId, namespace, name, payload) {
    // Construct the custom directive that needs to be sent
    // Gadget should declare the capabilities in the discovery response to
    // receive the directives under the following namespace.
    return {
        type: 'CustomInterfaceController.SendDirective',
        header: {
            name: name,
            namespace: namespace
        },
        endpoint: {
            endpointId: endpointId
        },
        payload
    };
};

/**
 * A convenience routine to add the a key-value pair to the session attribute.
 * @param handlerInput - the context from Alexa Service
 * @param key - the key to be added
 * @param value - the value be added
 */
exports.putSessionAttribute = function(handlerInput, key, value) {
    const attributesManager = handlerInput.attributesManager;
    let sessionAttributes = attributesManager.getSessionAttributes();
    sessionAttributes[key] = value;
    attributesManager.setSessionAttributes(sessionAttributes);
};

/**
 * To get a list of all the gadgets that meet these conditions,
 * Call the Endpoint Enumeration API with the apiEndpoint and apiAccessToken to
 * retrieve the list of all connected gadgets.
 *
 * @param {string} apiEndpoint - the Endpoint API url
 * @param {string} apiAccessToken  - the token from the session object in the Alexa request
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#call-endpoint-enumeration-api}
 */
exports.getConnectedEndpoints = function(apiEndpoint, apiAccessToken) {

    // The preceding https:// need to be stripped off before making the call
    apiEndpoint = (apiEndpoint || '').replace('https://', '');
    return new Promise(((resolve, reject) => {

        const options = {
            host: apiEndpoint,
            path: '/v1/endpoints',
            method: 'GET',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ' + apiAccessToken
            }
        };

        const request = Https.request(options, (response) => {
            response.setEncoding('utf8');
            let returnData = '';
            response.on('data', (chunk) => {
                returnData += chunk;
            });

            response.on('end', () => {
                resolve(JSON.parse(returnData));
            });

            response.on('error', (error) => {
                reject(error);
            });
        });
        request.end();
    }));
};

Alexa INI file

INI
Replace the blanks for the Amazon ID and Gadget Secret with the given secret from the Alexa Developer console. Use this in Visual Studio Code in a folder with the python file.
# Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
# 
# You may not use this file except in compliance with the terms and conditions 
# set forth in the accompanying LICENSE.TXT file.
#
# THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
# RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
# THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.

[GadgetSettings]
amazonId = Your_Alexa_ID
alexaGadgetSecret = Your_Alexa_Seceret

[GadgetCapabilities]
Custom.Mindstorms.Gadget = 1.0

Credits

Siddharth Agarwala

Siddharth Agarwala

1 project • 1 follower
Thanks to EV3 Dev and Amazon Alexa.

Comments