An AI HAT Trick

Chatbots have experienced a meteoric rise in popularity over the past few years, so it should come as no big surprise that hardware hackers have taken an interest in them as well. The tools work well enough as they are (and building and training a new model is no weekend project), so most of these hacks are all about upgrading the user interface. Typing away at a keyboard, or tapping out a message on a phone screen, does not make for a great chatbot experience. Talking to a little box that you can carry around in your pocket, in contrast, is much more natural.

We recently saw a very interesting example of a dedicated voice chatbot that was developed by PiSugar. However, this device relied on external APIs to work its magic, so if you are concerned about privacy, or want to use it where Wi-Fi is unavailable, it would be of no use. But now Jdaie Lin has shown how that same basic design can be modified to create a totally offline AI chatbot. It is not quite as small as the online version, but it is still small enough to carry around anywhere.

The chatbot is powered by a Raspberry Pi 5 single-board computer with 8GB of RAM. The computer is equipped with an active cooler because, while the Pi 5 can handle the algorithms, all that number crunching gets it hot enough to fry an egg. It is paired with a Whisplay HAT, which includes a display, microphone, speaker, and some buttons — essentially everything needed to make a decent voice assistant. For use on the go, Lin also attached a PiSugar 3 Plus 5,000mAh battery to keep the hardware up and running for an extended period of time.

In the video, Lin walks through the build process step by step. The basic architecture of these systems is pretty well a solved problem at this time — you need a speech recognition tool to transcribe voice requests, a large language model to process the requests, and a text-to-speech service to speak the responses returned by the model. Lin used the whisplay-ai-chatbot GitHub repository to easily hook these tools together, and specifically used Whisper for speech recognition, Ollama to deploy a Qwen3-1.7B model, and Piper for speech generation.

Demonstrations show that the voice assistant is pretty snappy for an offline chatbot. The user just presses a button, speaks their request, then an audible response is played through the speaker. The device can also run models in thinking mode for more complex questions, although the responses will be a bit more delayed.

If you have any interest in building your own offline chatbot, be sure to check out the video. It only takes a few hardware components, and Lin explains every detail needed to get the system up and running.