ReSpeaker Messenger

ReSpeaker Messenger: Sending and receiving Slack and Telegram messages using ReSpeaker Voice Interaction Board

My previous project using ReSpeaker was a Home Automation project to control different lights using your voice. This projects is about sending messages to Slack or Telegram using voice input and read out the message from the messenger. User can touch button 1 to send Slack message and touch button 7 to send Telegram message. For Telegram message, first you need to initiate the chat from Telegram client. This is because to send a message to Telegram you need the userid of the receiving person.

How it Works

This project uses Slack, Telegram, MQTT, hook.io and ReSpeaker.

In case of Slack messenger, We set an Outgoing Webhook to a hook.io microservice written in Node.js. This Node.js script receives the text and publish MQTT message with a specified topic. The Python script running on ReSpeaker subscribe this topic and upon receiving this message, the text is converted to audio using Microsoft Cognitive Computing Text to Speech API. This is read out using PyAudio.

In case of Telegram, the ReSpeaker python application use Telepot Python library to integrate Telegram API. Whenever a message is received the text is send to Microsoft Cognitive Service API and the receiving audio is played using PyAudio.

We have two applications running on the ReSpeaker, on Arduino Sketch and Python script. The Arduino Sketch receives touch event and send to the Python Script using Serial to start recording the voice input. When the start recording message is received, the script records the user's voice input and convert to text using Microsoft Cognitive Service Speech to Text API and send to the corresponding messenger depending on the touch button id. If the button touched is 1 then send to Slack messenger and if the touch button id is 7 then to Telegram messenger. Telepot library is used to send to Telegram messenger. For Slack, we have created a Slack Incoming Webhook. Below picture depicts working of the application.

How to Communicate from Arduino to Python

As you know ReSpeaker has two UART connections, "serial" and "serial1". The serial is connected to the USB port. The serial1 can be used to communicate from Arduino to the Python script. The serial1 is connected to the UART_RXD2 and UART_TXD2 of MT7688. So you can just open the Serial port /dev/ttyS2 and read the incoming data. The following test script opens the /dev/ttyS2 port and reads strings (Thanks to Jerry of Seeedstudio for suggesting this).

import serial
import io
port = '/dev/ttyS2'
baud = 57600
ser = serial.Serial(port, baud, timeout=1)
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser))
if ser.isOpen():
 print(ser.name + ' is open...')
while True:
 serialdata = sio.readline()
 print(serialdata)

Setting up ReSpeaker

Follow this document to get started with ReSpeaker

How to Use the Application

• Open Arduino IDE and load this file

• Connect your ReSpeaker to the network

• SSH to your ReSpeaker

• Create a folder to clone the project files

• Move to the folder and clone the repo https://github.com/krvarma/respeaker-messenger.git

• Got to respeaker-messenger folder

• Create an account or log on to Microsoft Cognitive Service Speech API](https://www.microsoft.com/cognitive-services/en-us/speech-api)

• Get an API Key and replace the variable BING_KEY in the Python Script Settings.py

• Log on to Slack.com, go to Settings->Add and app or integrations

• Go to Manage->Custom Integrations->Incoming Webhook. Create Incoming Webhook and copy the URL

• Replace the variable SLACK_WEBHOOK in the Python Script Settings.py

• Go to Slack->Manage->Custom Integrations->Incoming Webhook and create Outgoing Webhook. Copy the Token

• Log on to hook.io and create a new hook

• Copy the contents of this file to the hook

• Replace the access_token

• Log on to Telegram.org and create a new bot following this instructions

• Copy the token and replace the variable TELEGRAM_KEY in the Python Script Settings.py

• Install required Python package paho-mqtt using pip install paho-mqtt

• Install required Python package paho-mqtt using pip install telepot

• Install required Python package monotonic using pip install monotonic

• Run messenger.py using command line python messenger.py

• Open Telegram client application and send a simple 'hi' to any user, this will receive at the ReSpeaker end and the userid will be saved for sending future messages. This step is necessary because to send a message to Telegram, you need the userid of the user to whom you want to send the message.

• Touch the button 1 to send to messages to Slack and touch button 7 to send messages to Telegram.

• You can also send messages from Slack to the specified channel using the specified trigger work, this will be send to ReSpeaker and it will read out the text

If everything goes well, you have a working ReSpeaker Messenger and you can send messages using your voice. The application will read out the messages send from Slack or Telegram.

Demo Video

Slack Messenger https://www.youtube.com/watch?v=Q7amXfVdt4k

Telegram Messenger https://www.youtube.com/watch?v=_FrRhzkRTGc