Personal AI assistant with XiaoZhi AI

Background

As AI technology continues its rapid advancement, the integration of intelligent models onto low-level hardware architectures has become more relevant than ever. Inspired by the project 78/xiaozhi-esp32: An MCP-based chatbot | 一个基于MCP的聊天机器人 and xinnan-tech/xiaozhi-esp32-server: 本项目为xiaozhi-esp32提供后端服务，帮助您快速搭建ESP32设备控制服务器。Backend service for xiaozhi-esp32, helps you quickly build an ESP32 device control server., I have made the MCP Chatbot available onto W55MH32 boards using MicroPython.

Project Overview

As the diagram shows above, the speaker and microphone are shared by the same connection. Therefore, by default, the board uses speaker mode but when the record button is being hold, the board is in microphone mode.

You will also need internet access and run the scripts in order to connect to the AI server. Plug in an ethernet cable from your router to the board and plug in the USB port from the board to your PC.

As you start the script, the program should prompt you to start the connection by pressing the start button. Connection to the server will be started after pressing the button and now you should be able to communicate with the XiaoZhi AI. When the terminal asks you to type your message, you can now also access the microphone mode to talk to the agent.

Examples

Now, you can try talk to the AI using either Keyboard input from the terminal inside ThonnyIDE or by talking into the microphone.

Keyboard version:

Voice version:

P.S. This is one of the default template introduction in our own server for serving chatbot messages to the device.

You can also try asking him to introduce himself in other languages.

Keyboard version:

Voice version:

Moreover, you can talk to the Chatbot like you are talking with an LLM on your PC.

For example, in our own implementation of the xiaozhi-esp32-server, MCP Tool to get current weather is available and you could ask the AI about it.

Keyboard version:

Voice version:

It is recommended that you change to your desired language so that the AI can reply with the proper language.

Besides calling server MCP Tools, You can also add your own MCP Tool on your board so that it can interact with components on your board.

For example, I have made a MCP Tool function where when user ask the AI to turn on the LED based on the given pin.

Sample code on MCP Tool:

# Appending tool to its class member
self.tools.append(MCPTool('self.toggle_led', 'Turn onboard LED to on or off with values 1 as on, 0 as off.',
            {
             'type':'object',
             'properties': {
                 'value':{
                     'type':'integer',
                     'minimum':0,
                     'maximum':1
                     }
                 }
            }
        ))
        
# The actual function       
async def toggle_led(self, device, args = None):
        # Pin for LED
        pin = Pin('PB11', Pin.OUT)
        if args.get('value', None):
            pin.value(args.get('value'))
        else:
            pin.value(1-pin.value())
        text ='on' if pin.value() else 'off'
        return {'success':True, 'message': f'The LED is now {text}'}

Demo:

To create your MCP tool, it is important to know how to write the inputSchema for your MCP tool so that the AI can call your function properly. For more information, please have a look at Overview - Model Context Protocol.

Thank you for reading the article and I hope you find this tool interesting. If you are interested, a more detailed guide can be found on how to set up your own XiaoZhi Chatbot on the W55MH32 boards by going to W55MH32 and XiaoZhi Chatbot — W55MH32 MicroPython documentation. Hope you have a fun AI journey!