Published July 14, 2023

Breathing life into an Amazon Echo device!

Amazon's Alexa is boring, so we gave it some animatronic eyes and a CRT mouth. Now my kids won't leave it alone!

IntermediateFull instructions provided17,983

Things used in this project

Hardware components

5" B&W CRT television

The cosmetic condition of the device doesn't matter, as we will be removing it from the chassis anyway. Just make sure there's a nice, bright raster on the screen. The size of the TV's board will largely define the footprint of your project, as it is most likely the largest part of the build.

Amazon Alexa Echo Dot

M2, M3, M4 screw assortment w/nuts and washers

I used steel screws with a hex head (not countersunk). These are a great thing to have in your kit for all sorts of projects.

TDA7297 audio amplifier

Adafruit 16-Channel 12-bit PWM/Servo Driver

I used Adafruit's model for this because I know they make good products, but any similar servo board will do.

Audio Adapter, 3.5 mm Stereo Plug to 2x Sockets

Audio / Video Cable Assembly, 3.5mm Slim Stereo Plug to 3.5mm Slim Stereo Plug

Any old 3.5mm stereo cables will do for this. The shorter they are, they more easy they will be to manage. I ended up buying the connectors and making my own cables to the length I needed.

Arduino Mega 2560

You can use any microcontroller, as long as it can sense up to 5V on the sense pin (an Arduino Nano can only accept up to 3V)

Useful Sensors Person Sensor

4.5mm clear acrylic, laser cut to designs for the chassis

brass standoffs, assortment of M3 and M4 sizes

5V 8A switch-mode power supply

This should accept mains voltage for your region.

Hand tools and fabrication machines

3D Printer (generic)

Wire Stripper & Cutter, 32-20 AWG / 0.05-0.5mm² Solid & Stranded Wires

Soldering iron (generic)

Story

I believe that the secret to creating truly meaningful human-computer interaction is to create an emotional experience for the user. If we can make the user forget, even for just a few moments, that they are speaking with a computer—if we can inspire them to suspend their disbelief—then they will engage much more deeply with the machine.

Amazon's Alexa platform has been very successful with its voice functionality (it doesn't sound like a machine, after all). But I have a 3rd generation Amazon Echo, and I still can't get past the fact that I'm talking to a small black hockey puck. It always feels like I'm talking to machine, and a boring one at that. My goal for this project was to see if we can modify the Amazon Echo device to make it feel a bit more... alive!

The animatronic eyes

One of the most effective ways to breath life into any device is to add responsive, human-like eye contact. For our creature's eyes I used a 3D-printed animatronic eye mechanism designed by Will Cogley, which is a fast and relatively simple method for getting up and running with just a handful of parts. The eye movement and blinking is controlled by an Arduino and a 16-channel 12-bit PWM/Servo Driver from Adafruit. I originally used an Arduino Nano for this, but I accidentally destroyed it by sending it too much voltage, so I ultimately switched to an Arduino Mega 2560. (An Arduino Mega can analog sense up to 5V per pin, while an Arduino Nano can only handle 3V—more on this below.)

The eye mechanism.

The eyeballs were also 3D printed. I then sanded them and painted the iris pigmentation using acrylic paints. I originally made a silicone mold so that I could coat the eyeballs in a glossy resin (as recommended by this wonderful tutorial), but I had to change plans when I accidentally warped the shape of the eyeballs by applying too much heat (heat is necessary to purge air bubbles from the glossy resin). So instead I settled for applying a few drops of glossy resin only on the irises, which gave them just enough reflectivity to create a realistic effect. This small detail—a glint in the eye—is so important to give the illusion of life.

I used a dedicated 5V, 8A power supply to drive the servos. I originally tried to use a 3A power supply to drive the six SG90 servos, but it just wasn't powerful enough and let to a lot of servo jitter. After I changed power supplies, allocating 1A per servo, the jitters went away. I used a boost converter attached to the same 5V power supply to generate 8V for the Arduino.

The Person Sensor

I wanted the eyes to always maintain eye contact with the user, and to do this I used a Person Sensor to track the user's face. This was a very convenient solution that was almost "plug-and-play." This sensor requires SparkFun's Qwiic cabling system to connect, and I ended up buying a Qwiic cable pack that includes a Qwiic-to-jumper breakout cable so I could more easily integrate it with the Arduino. The Person Sensor works remarkably well for a $10 piece of kit, but its small design unfortunately doesn't include a convenient way to mount it to your build (there are no pin holes, for example). I ended up using a blob of Blu-Tack to stick it to the front of the animatronic eye module, which is fine for a prototype but not reliable in the longterm. A 3D printed mount that the sensor module could slide into would be a better solution.

I've been asked several times why I didn't just use a digital set of eyes on an LCD screen instead of the clunky, noisy animatronic mechanism. The answer is simple: I'm trying to inspire the user to forget that they are talking to a computer. Digital eyes on a screen would have been much simpler to build, but a real, moving, tangible set of 3D eyes does so much more to create the illusion of sentient life.

The user's first moment of contact

The first few seconds of contact between user and machine are so important for establishing the relationship, so I really wanted this to be a very powerful moment. To do this I programmed a "wake" sequence that would initiate when the user calls the creature's name: from a dormant state with closed eyes, the creature blinks to life, looks around, and then immediately engages the user with eye contact. The Alexa platform allows you to choose from a short list of "wake words, " so I changed the wake word from "Alexa" to "computer." I would have liked to customize this name, but that's just not possible yet with the Alexa platform.

Another major limitation of the Alexa platform was how it uses the wake word: you must first say the wake word and then give it a command. This is annoying, as I wanted the creature to wake up as soon as it hears its name, as any living creature would (I didn't want to have to say, "Computer, wake up!", but rather just "Computer!"). After some investigating, however, I realized that the LEDs on the Echo device light up as soon as it hears the wake word. After probing around inside, I found a point on one of the Echo's boards where the voltage drops from 2.5V to 1.1V when the LEDs illuminate.

This pad on the board showed us a voltage drop when Alexa hears the wake word (i.e., when the LEDs come on).

Soldering a sense line to the pad.

I soldered a thin jumper wire to this point and connected it to one of the analog pins on the Arduino, and also connected the Echo's ground to the Arduino's ground. I then added a few lines of code to the Arduino script so that it constantly monitors that voltage signal: if the voltage on that line drops below 2V, the Arduino then initiates the wake sequence for the eyes (if the eyes were in a dormant state). With this little soldering hack we were able to get our creature to respond to its name in a more organic way.

This hack worked well during testing with my Arduino Mega 2560, but when I was putting it all together for the final build I used a smaller Arduino Nano, and this posed problems. I couldn't figure out why the Arduino had stopped working. The reason was that when the Echo device is first powered on, it sends an inrush voltage on that same jumper line that is around 5V. An Arduino Nano can only handle 3V, so it fried the Nano completely. I thought about making a voltage divider circuit to reduce the voltage so I could continue using an Arduino Nano, but I was so eager to get our creature up and running that I just slapped in my Arduino Mega 2560 instead. This solved the problem, but visually wasn't quite as elegant.

The CRT voice visualizer

Another key element to making this device seem more "alive" is giving it a mouth that moves when it speaks. I'm a huge fan of CRT of televisions (you can check out some of my CRT restorations here and here), so for this project I converted a small 5" B&W CRT television into an audio waveform visualizer. If you want to learn how to do this, be sure to check out my separate tutorial on building a CRT audio waveform visualizer.

To get our creature's voice into the CRT, I plugged a 3.5mm splitter into the audio output port of the Echo device: one line connects to an amplifier and speaker, the other line connects to an amplifier and the CRT. To power these amplifiers I used a 12V boost converter attached to the 5V power supply. Each of the amplifiers has a potentiometer to adjust the output: for this project, one controls the volume of the sound coming out of the speaker, the other controls the amplitude of the waveform that we see on the CRT screen.

I completely removed the television board and CRT from its chassis. To conserve space, I also desoldered the turner module and the radio board from the main board of the television. These were both mounted vertically, so removing them allowed me to significantly reduce the size of the board. This would become very important later in the build.

Putting it all together

I struggled for a long time with finding a concept I liked for putting all of these disparate parts together:

5V power supply + boost converters
TV board + transformer
CRT screen
2x audio amplifiers + speaker
animatronic eye mechanism
Arduino + servo board
Amazon Echo device
power plug + switch + fuse

I ultimately decided to stack the build vertically, which would allow me to keep its countertop footprint to a minimum. But how to put it all together? I didn't want to hide it all inside an ugly, opaque enclosure, and I also wanted easy access to all the components for troubleshooting and repair. I decided to use a series of clear acrylic sheets, laser cut to designs that I created in Fusion 360, that could support the different layers of the build. I included screw holes in the designs so that I could then use an assortment of M3 and M4 brass standoffs as column structure between the layers. This worked well, though I made the mistake when ordering the acrylic sheets where some were 4.5mm thick, and others 5.5mm. This is important, because 4.5mm is just thin enough for the screw threads of the standoffs to poke out the other side, allowing me to continue the column structure across the sheets. 5.5mm was too thick for this, so for these I had to drill additional holes and use screws to anchor some of the standoffs. This was a good lesson—if you plan to use standoffs, use 4.5mm (or thinner) acrylic. I had bought an assortment of rubber feet some time back, so I used some of these on the bottom layer to get it up off the table surface.

I had originally wanted to remove the boards from the Amazon Echo device and use them without the outer case in order to save space, but I had trouble getting them to work correctly outside the Echo chassis for some reason (a grounding issue, perhaps?). I eventually lost patience and mounted the entire Echo device into my Alexatron chassis. This doesn't look very good, but since I already knew that I'll be moving to a GPT "brain" for future versions, we won't have to look at it for very long. I had also planned originally to power the 12V Echo device from the build's own 5V power supply (via 12V boost converter), but this also proved problematic for some reason, so I stuck with the Echo's native 220V-to-12V power supply.

In addition to the main power switch on the back, I added a separate switch that controls power for only the TV board and CRT. I did this so that I could work on Alexatron's other components without having to worry about the high voltage from the CRT (CRTs operate using thousands of volts). This also allows the user to disable the CRT if they want to listen to music from the Echo device for long periods.

Where do we go from here?

The obvious upgrade for MkII will be to get rid of the Alexa platform and instead adopt an AI platform such as ChatGPT. The trick will be keeping the voice interactivity as responsive as it is, which is something Alexa actually does very well. I want to keep the animatronic eyes, but they are quite fragile, so I'll look for a more stable mechanism for that, as well as better servos. If we can somehow reduce power consumption, we'll be able to eliminate most of the first layer of the chassis, which is taken up primarily by a huge 5V power supply and various boost converters to produce the different voltages.

From an interactivity standpoint, I'd like to add eyebrows. In the current configuration, the only way to project non-vocal emotion is through the eyelids (how wide or narrow they are), which is very limited. Adding eyebrows will open up a lot of new possibilities vis-a-vis non-verbal response.

I personally like the CRT "mouth"—it adds an element of nostalgia charm, which has resonated well with users. It would simplify the build considerably to replace it with an LCD solution, of course, but I just don't think it would be the same effect. I might try a smaller CRT (3" instead of 5"), which would also allow a smaller TV board.

Drop a comment below to tell me what you'd like to see in the next version!

Schematics

Code

Arduino code

#include <Wire.h>
#include <person_sensor.h>
#include <Adafruit_PWMServoDriver.h>

// How long to wait between reading the Person Sensor. The sensor can be read as
// frequently as you like, but the results only change at about 5FPS, so
// waiting for 200ms is reasonable.
const int32_t SAMPLE_DELAY_MS = 200;// default 200;

Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver();

#define SERVOMIN 140  // this is the 'minimum' pulse length count (out of 4096)
#define SERVOMAX 520  // this is the 'maximum' pulse length count (out of 4096)

int box_center_x;
int box_center_y;
int prev_center_x = 1;

unsigned long currentBlinkMillis = 0;
unsigned long previousBlinkMillis = 0;  // will store last time number changed
const long blinkInterval = random(5000,1200);   // period at which to change number (in milliseconds)
int blinkNumber = random(1,2);

unsigned long currentSleepMillis = 0;
unsigned long previousSleepMillis = 0;
const long sleepInterval = 20000; 

unsigned long loop_number = 0;

enum States {
  DORMANT,   // eyelids are closed, eyes are not moving
  AWAKE.     // eyelids are open, eyes are tracking faces
};

States State;

// our servo # counter
uint8_t servonum = 0;

int xval;
int yval;
int trimval;
int current_xval;
int prev_xval = 512;

int lexpulse;
int rexpulse;

int leypulse;
int reypulse;

int uplidpulse;
int lolidpulse;
int altuplidpulse;
int altlolidpulse;

int sensorValue = 0;
int outputValue = 0;
int switchval = 0;
int loopNumber = 0;

int ledPin = 3;
int sensorPin = A6;
unsigned long awakeTime = 30000;


void setup() {

  // You need to make sure you call Wire.begin() in setup, or the I2C access
  // below will fail.
  Wire.begin();
  pinMode(2, INPUT);
  Serial.begin(9600);
  pwm.begin();
  pwm.setPWMFreq(60);  // Analog servos run at ~60 Hz updates
  State = DORMANT;
  
  trimval = 500;   // this sets how wide the eyelids are positioned (higher number = wider eyes)
  trimval = map(trimval, 320, 580, -40, 40);
  uplidpulse = map(yval, 0, 1023, 400, 280);
  uplidpulse -= (trimval - 40);
  uplidpulse = constrain(uplidpulse, 280, 400);
  altuplidpulse = 680 - uplidpulse;

  lolidpulse = map(yval, 0, 1023, 410, 280);
  lolidpulse += (trimval / 2);
  lolidpulse = constrain(lolidpulse, 280, 400);
  altlolidpulse = 680 - lolidpulse;
  
  // Power up sequence to test eyes
  closeEyes();   
  delay(3000);
  wakeup();
  delay(1000);
  driftoff();
  delay(1000);

  Serial.println("setup complete");
  
}

void loop() {

  
  int sensorValue = analogRead(sensorPin); // reads for voltage change in Echo LEDs. LED's come on when Alexa hears wake word
  float voltage = sensorValue * (5 / 1023.0);
  Serial.println(voltage);

  if (voltage > 1.5 && State == DORMANT) {  // if asleep and no trigger word, Alexatron stays asleep
    State = DORMANT;
    Serial.println("DORMANT");
  } else if (voltage <= 1.5 && State == DORMANT) {  // if asleep and trigger word, Alexatron wakes up
    wakeup();

  } else if (voltage > 1.5 && State == AWAKE) {  // if awake and no trigger word, Alexatron stays awake for 10 seconds
    if(millis() - awakeTime > 0) {
        awake();
        awakeTime = millis();
    } else {
      driftoff();
    }
  } else if (voltage <= 1.5 && State == AWAKE) {   // if awake and trigger word, Alexatron stays awake
      awake();
    }   
  delay(50);

}

void blink() {   // script to execute one blink
  
  trimval = 550;   // this sets how wide the eyelids are positioned (higher number = wider eyes)
  trimval = map(trimval, 320, 580, -40, 40);
  uplidpulse = map(yval, 0, 1023, 400, 280);
  uplidpulse -= (trimval - 40);
  uplidpulse = constrain(uplidpulse, 280, 400);
  altuplidpulse = 680 - uplidpulse;

  lolidpulse = map(yval, 0, 1023, 410, 280);
  lolidpulse += (trimval / 2);
  lolidpulse = constrain(lolidpulse, 280, 400);
  altlolidpulse = 680 - lolidpulse;
  
  // closes eyelids
  pwm.setPWM(2, 0, 500);
  pwm.setPWM(3, 0, 240);
  pwm.setPWM(4, 0, 240);
  pwm.setPWM(5, 0, 500);

  delay(80);

  // opens eyelids to trimval value  
  pwm.setPWM(2, 0, uplidpulse);
  pwm.setPWM(3, 0, lolidpulse);
  pwm.setPWM(4, 0, altuplidpulse);
  pwm.setPWM(5, 0, altlolidpulse);
}

void awake() {
    
  Serial.println("AWAKE");
  /*   SENSOR TAKES READING   */
  person_sensor_results_t results = {};  // reads sensor
  if (!person_sensor_read(&results)) {
    Serial.println("No person sensor results found on the i2c bus");
    return;
  }

  for (int i = 0; i < results.num_faces; ++i) {
    const person_sensor_face_t* face = &results.faces[i];
    box_center_x = (face->box_left + ((face->box_right)-(face->box_left))/2);  // turns sensor box into x-axis point for eye-contact
    box_center_y = (face->box_bottom + ((face->box_top)-(face->box_bottom))/2);  // turns sensor box into y-axis point for eye-contact
    
    prev_center_x = box_center_x;

    // SERVO POSITIONING
    xval = ((box_center_x * -4) + 1023)*1.2;   // translate sensor information into servo info
    yval = ((box_center_y * -4) + 1023)*1.2;
    
    lexpulse = map(xval, 0, 1023, 220, 440);
    rexpulse = lexpulse;
    leypulse = map(yval, 0, 1023, 250, 500);
    reypulse = map(yval, 0, 1023, 400, 280);

    trimval = 550;   // this sets how wide the eyelids are positioned (higher number = wider eyes)
    trimval = map(trimval, 320, 580, -40, 40);
    uplidpulse = map(yval, 0, 1023, 400, 280);
    uplidpulse -= (trimval - 40);
    uplidpulse = constrain(uplidpulse, 280, 400);
    altuplidpulse = 680 - uplidpulse;

    lolidpulse = map(yval, 0, 1023, 410, 280);
    lolidpulse += (trimval / 2);
    lolidpulse = constrain(lolidpulse, 280, 400);
    altlolidpulse = 680 - lolidpulse;
    pwm.setPWM(0, 0, lexpulse);
    pwm.setPWM(1, 0, leypulse);

    /*  PERIODIC BLINKING  */
    unsigned long currentBlinkMillis = millis(); // store the current time
    if (currentBlinkMillis - previousBlinkMillis >= blinkInterval) { // check if interval has passed
      previousBlinkMillis = currentBlinkMillis;   // save the last time we changed number
      blink();
    }
  }
  delay(SAMPLE_DELAY_MS);
  State = AWAKE;
  
}

void wakeup() {   //  Creature wakes up from sleep (blinks and looks around)
  
    Serial.println("WAKEUP");  
    xval = 500;   // translate sensor information into servo info
    yval = 500;
    
    lexpulse = map(xval, 0, 1023, 220, 440);
    rexpulse = lexpulse;
    leypulse = map(yval, 0, 1023, 250, 500);
    reypulse = map(yval, 0, 1023, 400, 280);

    trimval = 650;   // this sets how wide the eyelids are positioned (higher number = wider eyes)
    trimval = map(trimval, 320, 580, -40, 40);
    uplidpulse = map(yval, 0, 1023, 400, 280);
    uplidpulse -= (trimval - 40);
    uplidpulse = constrain(uplidpulse, 280, 400);
    altuplidpulse = 680 - uplidpulse;

    lolidpulse = map(yval, 0, 1023, 410, 280);
    lolidpulse += (trimval / 2);
    lolidpulse = constrain(lolidpulse, 280, 400);
    altlolidpulse = 680 - lolidpulse;
    pwm.setPWM(0, 0, lexpulse);
    pwm.setPWM(1, 0, leypulse);

  pwm.setPWM(2, 0, uplidpulse);  //  opens eyelids
  pwm.setPWM(3, 0, lolidpulse);
  pwm.setPWM(4, 0, altuplidpulse);
  pwm.setPWM(5, 0, altlolidpulse);

  delay(100);

  blink();
  delay(500);
  blink();
  delay(500);

  pwm.setPWM(0, 0, 450); // eyes glance right
  delay(800);
  pwm.setPWM(0, 0, 220); // eyes glance left
  delay(1000);
  pwm.setPWM(0, 0, 330); // eyes look forward
  delay(1000);

  blink();
  delay(200);
  blink();

  State = AWAKE;
}



void driftoff() {   //  Slowly closes creature's eyes + eyeballs roll up
  
  pwm.setPWM(0, 0, 330);  // centeres eyes on x-axis
  // blink();
  //blink();
  for (int i = 1; i <= 50; i++) { // closes eyes slowly
    const double a = i / 50.0;
    pwm.setPWM(2, 0, uplidpulse + (400-uplidpulse) * (a)); 
    pwm.setPWM(3, 0, lolidpulse + (240-lolidpulse) * (a));
    pwm.setPWM(4, 0, altuplidpulse + (240-altuplidpulse) * (a));
    pwm.setPWM(5, 0, altlolidpulse + (400-altlolidpulse) * (a));
    pwm.setPWM(1, 0, 400 + (i)); // eyes roll up
    delay(40);
  }
  pwm.setPWM(2, 0, 460); // closes eyelids completely
  pwm.setPWM(3, 0, 240);
  pwm.setPWM(4, 0, 240);
  pwm.setPWM(5, 0, 460);
  delay(1000);
  State = DORMANT;
}




void closeEyes() {
  Serial.println("closeEyes start");
    xval = 500;   // translate sensor information into servo info
    yval = 500;
    
    lexpulse = map(xval, 0, 1023, 220, 440);
    rexpulse = lexpulse;
    leypulse = map(yval, 0, 1023, 250, 500);
    reypulse = map(yval, 0, 1023, 400, 280);

    trimval = 650;   // this sets how wide the eyelids are positioned (higher number = wider eyes)
    trimval = map(trimval, 320, 580, -40, 40);
    uplidpulse = map(yval, 0, 1023, 400, 280);
    uplidpulse -= (trimval - 40);
    uplidpulse = constrain(uplidpulse, 280, 400);
    altuplidpulse = 680 - uplidpulse;

    lolidpulse = map(yval, 0, 1023, 410, 280);
    lolidpulse += (trimval / 2);
    lolidpulse = constrain(lolidpulse, 280, 400);
    altlolidpulse = 680 - lolidpulse;
    pwm.setPWM(0, 0, lexpulse);
    pwm.setPWM(1, 0, leypulse);
    pwm.setPWM(2, 0, 460);
    pwm.setPWM(3, 0, 240);
    pwm.setPWM(4, 0, 240);
    pwm.setPWM(5, 0, 460);  


   delay(100);
     Serial.println("closeEyes finish");
}

// you can use this function if you'd like to set the pulse length in seconds
// e.g. setServoPulse(0, 0.001) is a ~1 millisecond pulse width. its not precise!
void setServoPulse(uint8_t n, double pulse) {
  double pulselength;

  pulselength = 1000000;  // 1,000,000 us per second
  pulselength /= 60;      // 60 Hz
  Serial.print(pulselength);
  Serial.println(" us per period");
  pulselength /= 4096;  // 12 bits of resolution
  Serial.print(pulselength);
  Serial.println(" us per bit");
  pulse *= 1000000;  // convert to us
  pulse /= pulselength;
  Serial.println(pulse);
}

Credits

Thomas Burns

4 projects • 44 followers

Father, maker, and lover of analog electronics. Check out more of my builds on my YouTube channel!

Breathing life into an Amazon Echo device!

Things used in this project

Hardware components

Hand tools and fabrication machines

Story

The animatronic eyes

The user's first moment of contact

The CRT voice visualizer

Putting it all together

Where do we go from here?

Custom parts and enclosures

Plans for chassis: base

Plans for chassis: Level 1

Plans for chassis: Level 2

Plans for chassis: Level 3

Schematics

ALEXATRON schematics

Code

Arduino code

Credits

Thomas Burns

Comments

Embed the widget on your own site

Breathing life into an Amazon Echo device!

Breathing life into an Amazon Echo device!

Things used in this project

Hardware components

Hand tools and fabrication machines

Story

The animatronic eyes

The user's first moment of contact

The CRT voice visualizer

Putting it all together

Where do we go from here?

Custom parts and enclosures

Plans for chassis: base

Plans for chassis: Level 1

Plans for chassis: Level 2

Plans for chassis: Level 3

Schematics

ALEXATRON schematics

Code

Arduino code

Credits

Thomas Burns

Comments

Related channels and tags