Okay Nordic, Light Switch!

Voice control for a Matter bulb

The goal of this project is to show how easy it is to create your own wake word and keyword models with Nordic Edge AI Lab: no dataset to record, no model code to write, just type a few phrases, train, and download a model that runs on the Axon NPU. The smart bulb is simply a fun way to see the result.

What it does?

I say "Okay Nordic" and then one of five commands. Light switch toggles the bulb, and scene one to scene four change its color. The wake word and the commands are recognized on the nRF54LM20B's Axon NPU, so the audio never leaves the chip. The DK then talks straight to the bulb with Matter over Thread, with no cloud and no hub automation in between.

How it works?

• A PDM microphone streams audio to the nRF54LM20B.
• Wake word: a model for "Okay Nordic" runs continuously on the Axon NPU.
• Commands: the wake word opens a short listening window, and a second model classifies what I say.
• A recognized command becomes a Matter command, and the DK sends it over Thread to the bulb it is configured to control.

Everything runs on the same SoC: the Axon NPU does the inference, while the application CPU runs the whole Matter stack (Matter API, Thread, and Bluetooth LE for commissioning). A binding is only a lookup table in the DK's configuration that says which device a command goes to. It is not used on the wire; Matter over Thread carries the message.

Step 1: The wake word, in one click

In Edge AI Lab the Wake Word page has two tabs: Train custom wake word and Use pre-built model (Okay Nordic). The pre-built model is already trained for the nRF54LM20B and its Axon NPU, so there are only three steps: select it, test it in the browser, and download the firmware package to run on the DK. Play Sample speaks the phrase, so you can check that it fits within one second.

Wake words are recognized in English only, and a custom phrase has to be 4 to 30 characters. If you train your own, the page's advice is to choose 1 to 3 words, avoid common phrases and keep it easy to pronounce.

Step 2: The five commands, from plain text

Open a keyword spotting solution (mine is called KWS) and go to Keyword Spotting Configuration. Each command is just a text phrase, and there is no dataset to record or upload:

• Type the phrase into Add new command.
• Press Play Sample to hear it and check that it fits within one second.
• Press Add, and repeat for each command. Commands are English only.

The Lab also offers a library of ready-to-use models if you want inspiration.

Besides my five phrases, the Lab adds two system classes, OTHER and SILENCE, so the model can say "none of these" instead of forcing a command. That gives seven outputs.

The right column below shows what the bulb does about a second after each command.

Then I started training. The Lab shows the progress and a timer, and in my case it took about 3 hours (the timer read 2 h 59 min) before the status changed to Training completed. The Results page then tells me what the model costs before I flash anything, and it has the Download model button (the model is built with Axon 1.5.0):

The Results page also lists what you need to run the model on a device: the trained model, the inference runtime (the nRF Edge AI Add-on), and the detection settings, which I cover next.

Step 3: Test and tune in the browser

Before touching any hardware, the Lab lets me test the model with my own microphone: press the microphone button, say a command, and watch what it detects. Two settings decide what counts as a detection, and both are set per command:

• Threshold: the probability a command has to exceed.
• Predictions in a row: how many consecutive predictions must stay above the threshold.

The Lab recommends starting low, and these are the values I used: 0.2 and 2 for the five commands, and 0 and 4 for OTHER and SILENCE. The page gives a simple rule for tuning: if it triggers when you didn't say the command, increase the threshold or the number of predictions in a row, and if it doesn't trigger when you do, decrease them. The same two numbers are then applied per keyword in the firmware.

Important: set these values for yourself in the firmware. The Lab's values are only a low, permissive starting point, and the firmware uses the numbers you put there, not the ones from the browser test. Left unchanged, they can make the model trigger far too often, including on speech that isn't a command. Tune them for your own room, microphone and voice (Step 6): raise the threshold or the predictions in a row if it triggers on the wrong words, and lower them if it misses real commands.

A quick look inside the Axon export

The download is a folder of generated C files. The model has seven outputs: the five commands plus OTHER and SILENCE, and the generated labels header shows their order.

The export README says the Axon can run inference up to roughly 15 times faster than the tflite-micro CPU engine, with the same nrf_edgeai API, which leaves the application CPU free for the Matter and Thread stack.

Step 4: Connect the microphone

Either PDM microphone works. The only differences are the wiring and the devicetree overlay you pass when building in Step 6.

Option A: Low-power PDM MEMS Microphone Fin

The Microphone Fin plugs straight onto the nRF54LM20 DK header, so there is nothing to wire. Press it onto the DK and build with the overlay boards/dmic_low_power.dtsi, which takes care of the pin routing.

Option B: Adafruit PDM microphone

The Adafruit board is powered from the DK's 1.8 V VDD:IO rail and connected with five jumper wires. Build with the overlay boards/dmic_adafruit.dtsi.

Step 5: Connecting the DK to the bulb

Matter only has to carry a command from the DK to the bulb, and the documented flow covers it, so I won't repeat every command here. There are two ways to set it up:

• CHIP Tool and a binding (the one I use). A bit harder, but perfectly fine. The DK and the bulb join the same Matter fabric and Thread network, the bulb gets an access-control entry that lets the DK control it, and the DK gets a binding for On/Off, Level Control and Color Control (clusters 6, 8 and 768). The binding is only configuration: it tells the DK where to send its messages.
• A commercial ecosystem, such as Apple Home. You map the actions to devices in the app, so no binding is needed, but it requires extra hardware.

Follow the Matter Edge AI generic switch sample documentation for commissioning, and the Matter: Light switch sample for the CHIP Tool commands that write the ACL and the binding. The add-on's wake word and keyword integration guide covers the keyword side. Wake word and keyword detection only start once the DK is commissioned.

After that, the DK talks to the bulb directly: in the log, each command shows a Notify Bounded Cluster line for every bound cluster, followed by a Matter InvokeCommandRequest. Note that "light switch" is a toggle, so it depends on the bulb's current state, while each scene sets an explicit color.

Step 6: Firmware

I started from the Matter Edge AI add-on's generic switch sample and only swapped in my own keyword model. The wake word stays "Okay Nordic". The commands in this project are just an example: they can be any phrases you train in the Lab. The steps stay the same, and the only rule is that everything follows the order of your model's labels. The add-on's wake word and keyword integration guide covers each step in detail.

What to update in the firmware

• Model files and getter: My keyword export replaces the generated files, and the getter is nrf_edgeai_user_model(). Guide: Adding exported models
• Keyword enum (keyword_app.h): 7 classes in the order of my labels header: OTHER, SILENCE, LIGHT_SWITCH, SCENE_FOUR, SCENE_ONE, SCENE_THREE, SCENE_TWO. Guide: Defining keyword classes
• Per-class settings (keyword_impl.c): The Lab’s values: 0.2 and 2 in a row for the five commands, 0 and 4 for OTHER and SILENCE. Guide: Keyword application hooks
• Keyword handler: One case per command: light switch toggles the bulb, each scene sets a color. Guide: Task callbacks
• Kconfig: The guide’s options, plus my export’s prj_example.conf. Guide: Kconfig options

With other commands, only the middle column changes: your own labels in your own order, your own settings, and one handler case for each command.

west build -b nrf54lm20dk/nrf54lm20b/cpuapp -- -DEXTRA_DTC_OVERLAY_FILE=boards/dmic_adafruit.dtsi
west flash --erase

Run it

Open the serial console and say "Okay Nordic", then a command. You should see Waiting for wakeword..., then wakeword detected, Looking for keywords, then Keyword detected: SCENE_TWO, 59 % (in my run the confidences were around 60 to 75%), and finally the Matter commands to the bulb. Additionally, the LEDs on the DK blink whenever the wake word is detected. Anything that isn't a command should do nothing.

What's next. The Lab's numbers describe the model on my desk. They say nothing about my room, other speakers or background noise, and a terminal only shows what scrolled past. Model Observability is a set of metrics computed directly on the device while the model runs. The Lab promises three things: catch it early when the model starts missing detections, find out whether the problem is the model or its conditions, and get recommendations on what to adjust. Next, I’m going to prepare a separate project that uses Model Observability to measure how the model behaves over time and under changing conditions.

Stay tuned!