Low latency voice control with PSOC™ Edge & Home Assistant

Motivation

If you have a few smart-home devices and a voice assistant in your home, you certainly know the pain: You ask it to turn on the light, but it takes multiple seconds until something happens! This is not surprising, since most well-known smart-home voice assistants are cloud-based.

So, after the wake word ("Ok,..." / "Hey,...") is detected, your voice is recorded, sent to a server, processed there, and the resulting action is sent back to the voice assistant, which then sends it to your smart-home hub (such as Home Assistant).

This technology is more than a decade old and is still around in many homes. Also, privacy can be a concern - many tech enthusiasts care about their data! So why would they want an always-on device listening in your living room and sending data to a remote server whenever it's triggered?

Especially when using Home Assistant, your whole smart home can run on your local network, without cloud services and - if you wish so - even without an internet connection.

Now, with PSOC™ Edge, you can easily build a voice assistant for your smart home that does not send anything to the cloud. It also does not even require an internet connection - everything happens locally on the edge, with stunning latency!

Keep in mind while reading the article, that this is a proof of concept - showing what's possible, but not (yet) enabling full stand-alone voice assistant functionality. But there's more to come! :)

Requirements

Before you can get started with this project, you need a Home Assistant instance set up (or any other smart home platform that supports API calls).

And you need at least one light that you want to control with the voice assistant in your Home Assistant instance.

API Key

We want to be able to control our Home Assistant entities directly from the voice assistant, so we will not opt for an MQTT-based approach with auto-discovery this time, but instead use the Home Assistant API to access the entities directly.

In your Home Assistant interface (usually at homeassistant.local:8123), navigate to the "Security" tab and scroll down to "Long-lived access tokens". Here you can create a token, we'll need it later :)

Setting up the PSOC™ Edge AI Kit

I use the PSOC™ Edge AI Kit (KIT_PSE84_AI) for this project.

Since the voice assistant light control application is one of the demo codes for the kit, we do not need to program anything ourselves - we do not even need to download a toolchain, thanks to the DevKit Experience.

So, head to the DevKit Experience page and select "Voice Assistant" and "Lighting Control", then connect your PSOC™ Kit and select the serial port of your PSOC™ device. Then you can deploy the application on your PSOC™ device.

Bridging PSOC™ Edge to Home Assistant

As you can see in the screenshot above, in step 3, you need to connect your device and click on the + button. You will then have this pop up for you, and you need to click on the device and hit connect (in our case, it is "Verbinden" in German).

Then step 3 will turn to something like this (see picture below), and you will need to flash your device as shown in the pictures below.

After successfully flashing your device, click on 'Experience Application' and you can test your application like this :

Controlling Desk Light

After trying the voice assistant with the board’s LED, let’s level up and control something useful: your desk lamp. The goal is simple—say a command, and your light switches on or off.

Setup:

• Plug your desk lamp into a smart plug.
• Add the smart plug to Home Assistant. Make sure it’s discovered and saved in your setup.
• Note the light’s entity ID in Home Assistant. You can find it under Overview >your device(in cour case light_buro)>parameter>Entity_Id (pictures below)

Code

We’ll use two small Python scripts to bridge your voice assistant and Home Assistant. The first script, ha_config.py, holds your Home Assistant IP address and Token:

# Home Assistant Configuration
# Update these values with your Home Assistant details

HA_URL = "http://192.168.178.26:8123/" # Your Home Assistant URL
HA_TOKEN = "xx.." # Long-lived access token from HA

Don't forget to adjust it.

Your Home Assistant URL is usually "homeassistant.local:8123/"

The token you will find only once after you save your Long-lived access tokens under PSOC™ Edge Voice Control, a one-time code will show to you that you will need to copy (that's the token code).

The second script, edge-to-ha.py, does the actual talking between your board and Home Assistant. Update the serial port to match your board and set the light’s entity ID to the one you noted earlier.

Once those two details are correct, you’re basically ready to roll. The rest of the code is straightforward, but if anything’s unclear, leave a comment, and we’ll help you troubleshoot.

import serial
import re
import requests
from ha_config import HA_URL, HA_TOKEN

# Serial port configuration
SERIAL_PORT = 'COM34'
BAUD_RATE = 115200

# Home Assistant entity
LIGHT_ENTITY = "light.light_buro"

# Home Assistant API client
class HomeAssistantAPI:
    def __init__(self, url, token):
        self.url = url.rstrip('/')
        self.headers = {
            "Authorization": f"Bearer {token}",
            "Content-Type": "application/json",
        }
    
    def call_service(self, domain, service, entity_id):
        """Call a Home Assistant service"""
        endpoint = f"{self.url}/api/services/{domain}/{service}"
        data = {"entity_id": entity_id}
        
        try:
            response = requests.post(endpoint, headers=self.headers, json=data, timeout=5)
            response.raise_for_status()
            print(f"Success: {domain}.{service} called on {entity_id}")
            return True
        except requests.exceptions.RequestException as e:
            print(f"Error calling HA service: {e}")
            return False
    
    def turn_on_light(self, entity_id):
        """Turn on a light"""
        return self.call_service("light", "turn_on", entity_id)
    
    def turn_off_light(self, entity_id):
        """Turn off a light"""
        return self.call_service("light", "turn_off", entity_id)

# Initialize Home Assistant API clientg
ha_api = None

# Intent handler functions
def TurnOnLight():
    """Handler for TurnOnLight intent"""
    print("Executing TurnOnLight...")
    if ha_api:
        ha_api.turn_on_light(LIGHT_ENTITY)

def TurnOffLight():
    """Handler for TurnOffLight intent"""
    print("Executing TurnOffLight...")
    if ha_api:
        ha_api.turn_off_light(LIGHT_ENTITY)

# Intent mapping
INTENT_HANDLERS = {
    'TurnOnLight': TurnOnLight,
    'TurnOffLight': TurnOffLight,
}

def parse_intent(line):
    """Parse intent name from serial output"""
    # Look for lines like "Intent name: TurnOnLight"
    match = re.search(r'Intent name:\s*(\w+)', line)
    if match:
        return match.group(1)
    return None

def main():
    """Main function to read from serial and execute intent handlers"""
    global ha_api
    
    # Initialize Home Assistant API
    print("Initializing Home Assistant API...")
    ha_api = HomeAssistantAPI(HA_URL, HA_TOKEN)
    print(f"Home Assistant URL: {HA_URL}")
    
    print(f"\nConnecting to {SERIAL_PORT} at {BAUD_RATE} baud...")
    
    try:
        # Open serial connection
        with serial.Serial(SERIAL_PORT, BAUD_RATE, timeout=1) as ser:
            print("Connected! Listening for intents...")
            
            while True:
                # Read line from serial port
                if ser.in_waiting > 0:
                    line = ser.readline().decode('utf-8', errors='ignore').strip()
                    
                    if line:
                        print(f"Received: {line}")
                        
                        # Parse intent from line
                        intent_name = parse_intent(line)
                        
                        if intent_name:
                            # Execute corresponding handler if it exists
                            handler = INTENT_HANDLERS.get(intent_name)
                            if handler:
                                print(f"Intent detected: {intent_name}")
                                handler()
                            else:
                                print(f"Unknown intent: {intent_name}")
                    
    except serial.SerialException as e:
        print(f"Serial port error: {e}")
    except KeyboardInterrupt:
        print("\nStopped by user")
    except Exception as e:
        print(f"Error: {e}")

if __name__ == "__main__":
    main()

Code Output

When you run it, use “OK Infineon” as the wake word and try simple commands like “turn on the desk light” or “turn off the desk light.” You’ll see the status update in Home Assistant and, of course, your lamp will respond in real time.

There’s a live demo video so you can see it in action!

Farewell

Thanks for sticking all the way to the end! I hope this project gave you a fun peek into what is possible with the new PSOC™ Edge AI Kit. If you try it yourself or decide to push it further, let us know! Until then 🤖💡💡