Pocket-sized AI voice assistant powered by Pico 2 W

I built a pocket-sized AI voice assistant using a Raspberry Pi Pico 2 W, Zephyr RTOS, and the OpenAI Realtime API.

The device uses a simple push-to-talk interface: press a button, speak through the microphone, and receive a natural AI voice response through the speaker.

Wiring

The Raspberry Pi Pico 2 W is connected to the ES8311 audio codec through I2S and I2C.

ES8311                 Raspberry Pi Pico 2 W
                             Micro-USB
                         +-------+-------+
PA_EN   <--- GP0    1 o |               | o 40 VBUS
DSDIN   <--- GP1    2 o |               | o 39 VSYS <--- BATTERY +
GND     ---- GND    3 o |               | o 38 GND  ---- BATTERY -
DSOUT   ---> GP2    4 o |               | o 37 3V3_EN
MCLK    <--- GP3    5 o |               | o 36 3V3 ---> VCC
BCLK    ---> GP4    6 o |               | o 35 ADC_VREF
LRCK    ---> GP5    7 o |               | o 34 GP28
               GND  8 o |               | o 33 AGND
SDA     <--> GP6    9 o |               | o 32 GP27
SCL     <--> GP7   10 o |               | o 31 GP26
               GP8 11 o |               | o 30 RUN
               GP9 12 o |               | o 29 GP22
               GND 13 o |    RP2350     | o 28 GND
              GP10 14 o |               | o 27 GP21
              GP11 15 o |               | o 26 GP20
              GP12 16 o |               | o 25 GP19
              GP13 17 o |               | o 24 GP18
               GND 18 o |   CYW43439    | o 23 GND  ---- UART GND
              GP14 19 o |               | o 22 GP17 <--- UART TX
              GP15 20 o |               | o 21 GP16 ---> UART RX
                         +---- antenna ----+

How it works

The device is built around the Raspberry Pi Pico 2 W, running Zephyr RTOS.

The Pico 2 W does not have a dedicated I2S peripheral, so the I2S protocol is implemented using PIO (Programmable I/O) to achieve reliable realtime audio streaming.

For realtime voice interaction, the device uses a WebRTC realtime audio connection with the OpenAI Realtime API.

To improve performance, the dual-core architecture of the Pico 2 W is utilized:

• Core 1: Handles audio capture, playback, I2S communication, and audio encoding/decoding
• Core 0: Handles WebRTC communication, networking, and AI voice data exchange

The complete audio pipeline:

Microphone
    |
ES8311 Audio Codec
    |
Raspberry Pi Pico 2 W
    |
Zephyr RTOS
    |
WebRTC Realtime Audio Connection
    |
OpenAI Realtime API
    |
Speaker

What's Next

I’d like to take this project further by moving from push-to-talk toward always-on voice interaction, with wake-word detection and improved power efficiency. I also want to explore more of Zephyr RTOS on this platform, including power management, peripherals, and local processing. With a smaller form factor and a battery-powered design, this could eventually become a compact wearable device for natural, hands-free AI interaction.