Espresso Yourself with Your Voice

With computing equipment getting smaller and cheaper and appearing in everything from smart watches to coffee machines these days, human-computer interactions are becoming more casual. Plugging a keyboard and mouse into your smart speaker to turn on the lights, or your phone to write an email, wouldn’t make any sense at all. For this reason, touchscreens and voice control systems have become the dominant user interfaces in this market segment.

Touchscreens and voice control systems are very different, each with its own pros and cons. Some people love the convenience of speaking their requests from across the room, while others are frustrated with the so-so accuracy and lack of privacy associated with voice control. Whatever your preference may be, one thing is certain: you get what you get. If the device you want only comes with a touchscreen, then voice control will never be an option.

Electronics hobbyist Ed Holmes picked up a very fancy Franke A600 coffee machine that comes equipped with a touchscreen for control. But Holmes did not want to have to use a touchscreen in the kitchen where hands-free operation would be better. So rather than live with what he saw as a blatant case of design malpractice, Holmes got busy hacking away at the machine to teach it to understand voice commands.

Rather than gut the machine and replace its hardware, Holmes came up with a creative solution that allowed him to add a few components that piggyback on top of its existing functionality. He used an STM32WB55 USB dongle development board hooked up to a microphone. The microcontroller was then flashed with custom firmware that leverages Picovoice for voice recognition.

That got Holmes half of the way there, but there was still the matter of causing these voice commands to control the coffee machine. This was accomplished by connecting the development board to the coffee maker via USB. That allowed him to send customized USB HID packets to it, which simulate touch events on the existing screen. By mapping out the coordinates of all of the items on the screen in advance, Holmes could send packets specifying a touch event that matches up with the request made in the voice command.

Pretty cool solution, huh? If you also have a coffee maker (or other kitchen appliance) with as much computing power as your desktop computer, then you might want to give this project, called Speech2Touch, a look. Source code and build instructions have been released under a permissive MIT license to help your own appliances become better listeners.