Published November 15, 2019 © CC BY-NC-SA

Ted The Talking Toaster

Conversations with a toaster have always been very one sided. Until now, meet Ted the talking Toaster!

AdvancedFull instructions providedOver 1 day2,368

Things used in this project

Hardware components

Raspberry Pi 3 Model B Starter Kit

Google AIY Voice

SG90 Micro-servo motor

Raspberry Pi Camera Module

Toaster

Pipe Cleaner

Software apps and online services

Raspberry Pi Raspbian

Google Dialogflow

Google Cloud Platform

Hand tools and fabrication machines

Drill

Soldering Iron Kit

Hot glue gun

Screwdriver

As an Amazon Associate I earn from qualifying purchases you make using my affiliate links.

Story

Overview

Below an overview of the content:

Project video
Toaster
Electronics
Chat/Voice bot
Remo.tv
World domination

Project Video

Toaster

To make a talking toaster the first thing we need is, a toaster. I got this one from a friend, but thrift stores are a great place to find yourself a broken toaster with good looks.

The original toaster

The first thing we're going to do is remove as much of the internals as possible, while keeping the outside looking nice.

Removing the internals

Some fiddling and the right amount of violence is needed to make this happen, but in the end you'll have plenty of space.

The cleared toaster

Electronics

With all that space, we can start filling it up. Our aim is to make a talking toaster, so that's what we wil start with.

We're going to use a Raspberry Pi 3 b+ with a Google AIY kit. This kit consists of a hat, microphone and speaker. They provide a great tutorial on how to set it all up.

For this project we've used an older version of the kit, but the newer ones will work as well.

Google AIY Kit

This covers the talking hardware for our little bread burner, but we also want him to express himself. To make this happen we're going to drill two holes in one of the front panels.

Drilled holes

Through these holes we put two little servo's, they're the start of his eyebrows.

Servo glued to the inside of the toaster

Now we can wire the servo to the AIY hat, first we need to solder some pins and then we can easily attach them. There's some more info on the possible extensions below.

Hat schematics

The last offline step is to glue on some fuzzy pipe cleaners as eyebrows and googly eyes as, you guessed it, his eyes.

Looking good

And with that, we've got a very good looking little heating element. He looks the part, so he deserves a name, let's call him Ted.

Chat/Voice bot

At this point you can keep following the AIY tutorial to make the toaster your personal google assistant, or write your own custom code. In this case we opt for creating a custom chatbot, using Dialogflow. They also provide a good tutorial on how to setup a chatbot.

After some creative writing our chatbot is done, but at this point Ted is just a toaster and the chatbot doesn't do much. To change this we open up the Dialogflow chatbot API. This means we can access the chatbot from our Raspberry Pi. And now the fun begins The flow of data is as followed:

Data flow just Ted

1
The microphone picks up someone speaking and saves the audio

2-3
Using some Google magic (Speech-To-Text) we extract the text from the audio

4-5
This text is sent to our chatbot (Dialogflow) and is matched with an intent.After which one of the possible answers is send back to the Raspberry Pi (Ted)

6-7-8
Using Text-To-Speech, the text is converted to audio and played via the speaker

You can find the full code for this attached to the article.

Ted can now listen and talk, but how is he going to show the right emotion for each response?

When fetching the response from the Dialogflow API, it also sends back the intent name. Ending every intent with "_emotion" makes sure we can extract the emotion linked with each response and move the servos accordingly.

Teds emotions

Example, our intent is called "climateChange_angry" he will extract everything after the "_".This means "angry", and then moving the servo to look angry.

Now Ted can listen, talk and look frisky, that's one small step for Ted, one giant leap for toasterkind.

Remo.tv

Ted might be a tad evil, who would think a toaster could be out for world domination? As his creators we're responsible for his well being, and since he demanded to be connected to the internet to spread his message, let's do exactly that.

Remo.tv

It turns out we're not the only ones who want to connect robots to the internet, Remo.tv is way ahead of us. This means that all we have to do is attach a Pi Camera and follow their instructions.

Ted on Remo

Now people can see Ted, but talking to him is tricky. However, Remo.tv allows you to implement your own chat handeling. So removing some parts of our previous code and adding the custom chat handler does the trick. Now the data flow is:

Remo Data Flow

1
Chat message is received from Remo.tv

2-3
This message is sent to Dialogflow and the response with emotion is returned.

6-7
The response text is converted to audio via the Text-To-Speech.

8
The audio is played via the speaker and immediately picked up by the microphone.

9
This audio is sent to Remo to be broadcast to everyone who's tuned in.

World domination

Toast'en

All set, Ted can start convincing the world of his vision, and we now feel like some toast.Would you like to be converted, just follow this link.

With that, all that's left to say is: #Toaster2020!

Code

# Generic imports
import time
import os

# Dialogflow setup
import aiy.voice.tts 
import dialogflow_v2 as dialogflow
from aiy.board import Board
from aiy.cloudspeech import CloudSpeechClient

session_client = dialogflow.SessionsClient()
session = session_client.session_path('xxxxxxx', 12345)

# Servo setup
from gpiozero import Servo
rightEyebrowServo = Servo(26) 
leftEyebrowServo = Servo(24)


def getDialogResponse(text):

    text_input = dialogflow.types.TextInput(text=text, language_code='EN')

    query_input = dialogflow.types.QueryInput(text=text_input)

    response = session_client.detect_intent(session=session, query_input=query_input)

    text = response.query_result.fulfillment_text

    return response

def say(test):
    aiy.voice.tts.say(test, lang='en-US', volume=30, pitch=20, speed=80, device='default')

def angry():
    rightEyebrowServo.min()
    leftEyebrowServo.max()

def neutral():
    rightEyebrowServo.mid()
    leftEyebrowServo.mid()

def sad():
    rightEyebrowServo.max()
    leftEyebrowServo.min()

def sceptic():
    rightEyebrowServo.mid()
    leftEyebrowServo.max()

def frisky():
    neutral()
    time.sleep(0.1)
    for i in range(5):
        angry()
        time.sleep(0.1)
        sad()
        time.sleep(0.1)
    neutral()

def testEmotions():

    say("Angry")
    angry()
    time.sleep(2)

    say("Neutral")
    neutral()
    time.sleep(2)

    say("Sad")
    sad()
    time.sleep(2)

    say("sceptic")
    sceptic()
    time.sleep(2)

    say("frisky")
    frisky()
    time.sleep(2)


angry()

print('setup...')
client = CloudSpeechClient()

while True:

    print('Listening...')
    spokenText = client.recognize() 

    if spokenText is None:
        print('You said nothing.')
    else:
        response = getDialogResponse(spokenText)
        responseText = response.query_result.fulfillment_text
        responseIntent = response.query_result.intent.display_name
        responseIntentEmtion = responseIntent.split("_",1)[1].lower()

        if responseIntentEmtion == 'sceptic':
            sceptic()
        elif responseIntentEmtion == 'angry':
            angry()
        elif responseIntentEmtion == 'neutral':
            neutral()
        elif responseIntentEmtion == 'sad':
            sad()
        
        say(responseText)
        if responseIntentEmtion == 'frisky':
            frisky()

        angry()
    
    time.sleep(2)

# Generic imports
import time
import os
import re

# Dialogflow setup
import aiy.voice.tts 
import dialogflow_v2 as dialogflow

session_client = dialogflow.SessionsClient()
session = session_client.session_path('xxxx', 12345)

# Servo setup
from gpiozero import Servo
rightEyebrowServo = Servo(26) 
leftEyebrowServo = Servo(24)

#Ted's custom functions
def getDialogResponse(text):

    text_input = dialogflow.types.TextInput(text=text, language_code='EN')

    query_input = dialogflow.types.QueryInput(text=text_input)

    response = session_client.detect_intent(session=session, query_input=query_input)

    text = response.query_result.fulfillment_text

    return response

def say(test):
    aiy.voice.tts.say(test, lang='en-US', volume=15, pitch=20, speed=80, device='default')

def angry():
    rightEyebrowServo.min()
    leftEyebrowServo.max()

def neutral():
    rightEyebrowServo.mid()
    leftEyebrowServo.mid()

def sad():
    rightEyebrowServo.max()
    leftEyebrowServo.min()

def sceptic():
    rightEyebrowServo.mid()
    leftEyebrowServo.max()

def frisky():
    neutral()
    time.sleep(0.1)
    for i in range(5):
        angry()
        time.sleep(0.1)
        sad()
        time.sleep(0.1)
    neutral()

def testEmotions():

    say("Angry")
    angry()
    time.sleep(2)

    say("Neutral")
    neutral()
    time.sleep(2)

    say("Sad")
    sad()
    time.sleep(2)

    say("sceptic")
    sceptic()
    time.sleep(2)

    say("frisky")
    frisky()
    time.sleep(2)

# example for adding a custom chat handler
main_chat_handler = None


# Reboot function, to be added to the extended chat commands.
def reboot(*args):
    os.system('sudo shutdown -r now')
    exit()  

def setup(robot_config, chat_handler):
    global main_chat_handler
    
    # Any chat related setup code goes here


    
    # Add a reboot command to the extended chat commands
    if robot_config.getboolean('tts', 'ext_chat'):
        import extended_command
        extended_command.add_command('.reboot', reboot)    
    
    main_chat_handler = chat_handler
    
    
def handle_chat(args):

    # Your custom chat handling code goes here
    spokenText = 'internal_timed_info'
    try:
        spokenText = args['message']
    except:
        print('introduce yourself now')

    response = getDialogResponse(spokenText)
    responseText = response.query_result.fulfillment_text
    responseIntent = response.query_result.intent.display_name
    responseIntentEmtion = responseIntent.split("_",1)[1].lower()

    if responseIntentEmtion == 'sceptic':
        sceptic()
    elif responseIntentEmtion == 'angry':
        angry()
    elif responseIntentEmtion == 'neutral':
        neutral()
    elif responseIntentEmtion == 'sad':
        sad()


    print('You said ' + spokenText)
    print('intent: ' + responseIntent)
    print('ted says: ' + responseText)
    
    say(responseText)

    if responseIntentEmtion == 'frisky':
        frisky()

    angry()
    time.sleep(0.2)
    rightEyebrowServo.detach()
    leftEyebrowServo.detach()
     
    # Call the main chat handler
    main_chat_handler(args)

Ted The Talking Toaster

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Overview

Project Video

Toaster

Electronics

Chat/Voice bot

Remo.tv

World domination

Schematics

Ted Data flow

Remo Data Flow

Code

Ted Code

Remo Code

Credits

Comments

Embed the widget on your own site

Ted The Talking Toaster

Ted The Talking Toaster

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Overview

Project Video

Toaster

Electronics

Chat/Voice bot

Remo.tv

World domination

Schematics

Ted Data flow

Remo Data Flow

Code

Ted Code

Remo Code

Credits

Comments

Related channels and tags