DEX – An Interactive AI Kiosk for Restaurants

Overview

Ordering food in public spaces is becoming increasingly digital, but current touchscreen kiosks still fall short in one critical area: interaction. They may display glossy pictures and responsive buttons, but they don’t understand your needs, preferences, or questions. DEX changes that.

DEX is not your average kiosk. It's a prototype AI-powered assistant designed to assist you in making better food decisions—whether it's asking about ingredients, understanding dietary preferences, or just asking, "What goes well with an Aloo Tikki burger?" It talks. It listens. It responds—and it does so in multiple languages.

The Idea Behind DEX

Most digital kiosks are made for tapping and ordering. But what about discovering? Exploring? Understanding your food before you choose it? DEX was built on that gap.

Think of DEX as your interactive, voice-enabled food guide. It's especially helpful in places where language barriers, dietary restrictions, or information gaps stand in the way of good decisions.

Core Features

• Voice Interaction: Users can ask DEX anything via voice—no typing or tapping needed.
• Multilingual Support: DEX understands and responds in multiple languages, making it accessible to a diverse range of customers.
• Food Intelligence: Ask about calories, ingredients, or get smart suggestions based on your current choice.
• ESP-NOW Communication: Voice input is captured by a XIAO ESP32S3 and sent to the MaTouch ESP32-S3 display wirelessly.
• Cloud-Powered AI: DEX uses ElevenLabs for speech-to-text and OpenAI APIs to respond intelligently to user queries.
• Audio Response: With the MAX98357A I2S amplifier, DEX speaks its responses clearly, like a real assistant.
• Touch-Driven Interaction: You can also browse the menu visually and tap on any item—DEX will speak about it, giving you details and recommendations.

Hardware Used

• Seeed Studio XIAO ESP32S3 Sense – Captures voice using built-in mic.
• MaTouch ESP32-S3 Display – Serves as the main user interface with visual feedback.
• MAX98357A I2S Amplifier – Outputs clear, natural-sounding voice responses.
• Micro SD Card – For saving temporary audio recordings.

How It Works

• User holds down a button on the XIAO ESP32S3 to record a query.
• The audio is stored as a WAV file and sent to the ElevenLabs API.
• Transcribed text is sent via ESP-NOW to the MaTouch display.
• OpenAI's chat API processes the query and generates a meaningful response.
• The response is spoken back using OpenAI TTS and played through the speaker.
• Alternatively, users can tap on any menu item on screen to hear DEX describe it.

Why This Matters?

DEX is especially useful in multilingual environments like international airports, food courts, or tourist-heavy areas where understanding a menu isn’t always easy. It's more than a tool—it's an experience-enhancer.

And while this prototype uses a smaller MaTouch display, the concept is scalable. The full product could integrate with larger kiosk displays for a more immersive and commercial-ready experience.

Conclusion

With DEX, we’re not just upgrading kiosks; we’re redefining them. This is not a voice gimmick or a fancy UI. It’s a meaningful assistant that empowers users with clarity, confidence, and choice—in any language, on any menu.

Let’s make kiosks smarter. Let’s make menus talk.

DEX doesn’t take your order. It helps you make it.