Pollexy: A Special Needs Voice Assistant

Children and adults with autism have a wide range of needs, and as such there’s currently no single solution that helps them with their day-to-day needs. 16-year-old Calvin Larson is one of the many people with autism spectrum disorder (ASD). And while he is able to respond to verbal instruction, he is not able to speak responsively. Many tasks, such as going to the bathroom or brushing is teeth, require a human to verbally prompt him along the way.

Calvin’s father, Troy Larson, works as a Senior DevOps Cloud Architect at Amazon Web Services, and has been to help him navigate some of these challenges. Since Calvin reacts well to verbal cues, the Pollexy Project aims to leverage Amazon’s Polly text-to-speech technology and its Lex conversational interface into a Raspberry Pi-based system that can assist him with some of these tasks without human interaction.

As Larson writes:

I setup a Raspberry Pi in his room and closed his door and crouched around the corner with other family members so Calvin couldn’t see us. I connected to the Raspberry Pi and instructed Polly to speak in Joanna’s familiar pacific tone, “Calvin, it’s time to take a potty break. Go out of your bedroom and go to the bathroom.” In a few seconds, we heard his doorknob turn and I poked my head out of my hiding place. Calvin passed by, looking at me quizzically, then went into the bathroom as Joanna had instructed. We all looked at each other in amazement — he had listened and responded perfectly to the completely invisible voice of someone he’d never heard before.

You can read more about this amazing voice assistant project — powered by Amazon Polly and Raspberry Pi — on the AWS blog here.