Speech recognition for self-driving toy car on Jetson

We saw this Self-Driving Toy Car article and thought it would be fun to make it even more intelligent by controlling it with voice. Picovoice can be used through multiple SDKs on Jetson Nano. Let us go with Python and make our voice interface ready in 3 steps:

0. Set up your NVIDIA Jetson Nano

If you haven't done it yet, refer to the “Getting Started” page.

1. Configure your microphone

As for the microphone, there are already several expansion boards off the shelf for Jetson Nano, but I am going with a simple USB mic.

Connect the microphone and get the list of available input audio devices by running the following command inside your terminal:

arecord -L | grep plughw

The output should be similar to below:

plughw:CARD=PCH,DEV=0

Copy this line and create a .asoundrc file in your home folder with these options:

pcm.!default {
   type asym
   capture.pcm "mic"
}
pcm.mic {
   type plug
   slave {
      pcm "${INPUT_AUDIO_DEVICE}"
   }
}

Replace ${INPUT_AUDIO_DEVICE} with what you copied earlier. You may need to reboot the system for these settings to take effect. It is also a good idea to go to Setting->Sound->Input and check if everything is working fine and that Linux identifies your microphone correctly.

2. Install Picovoice Package

First you need to install some dependencies using the following command:

sudo apt-get install python3-pyaudio portaudio19-dev libffi-dev

After that, grab the Picovoice python package:

sudo pip3 install picovoice picovoicedemo

The picovoicedemo package lets test our models rapidly on Jetson Nano.

2. Create Context model

Go to Picovoice Console, an online, interactive tool to easily design models for both wake word engine (Porcupine) and speech-to-intent engine (Rhino).

Here's the simple context, which understands changing speed (“Rate”) and turning on/off (“State”). You can copy the paragraph below and import it into Picovoice Console using the Import YAML button:

context:
  expressions:
    changeSpeed:
      - Slow $Rate:rate
      - Speed $Rate:rate
      - $Rate:rate (the) speed
      - Set (the) speed to $pv.SingleDigitInteger:speed
    changeState:
      - "[switch, turn] $State:state (the) engine"
  slots:
    State:
      - On
      - Off
    Rate:
      - down
      - up
      - increase
      - decrease

After training the model, download and extract it into your home folder. You can train your custom wake word or pick “computer” as the wake word from Picovoice GitHub.

3. Add it to your self-driving toy car project

Thus far, we have created and tested our voice interface. Next, we will add it to a self-driving toy car project. Use the following Python code with the aforementioned Picovoice SDK for Python - Python API documentation.