CraveGuide: Voice-Driven Food Recommendation & Ordering

When people visit a restaurant, choosing what to eat is not always as simple as looking at a menu. A customer may know what they are craving—something spicy, sweet, sour, salty, vegetarian, or non-vegetarian—but may not know which dish matches that craving. They often have to browse through several menu pages, read descriptions, compare dishes, and ask for recommendations. This becomes even more difficult when the menu is large or when the customer wants to try something new but cannot decide what would suit their taste.

CraveGuide was developed to make this experience simpler and more natural. Instead of searching through the menu, the user can simply speak their craving, such as “I want something spicy and vegetarian.” The device converts the voice input into text, identifies the relevant preferences, and presents matching dishes on an e-paper display. The user can then navigate through the recommendations using physical buttons, select a dish, choose the required quantity, and place the order.

The main objective of CraveGuide is therefore to create a simple, interactive, voice-driven food recommendation and ordering experience. The system focuses on understanding basic taste and dietary preferences, providing relevant menu recommendations, reducing the effort required to browse a menu, and making the final selection and ordering process quick and intuitive.

Hardware Requirements

CraveGuide is built around a compact set of embedded hardware selected to support low-power operation, voice interaction, portable use, and a simple physical interface. A major advantage of the project is that most of the core hardware comes from the Seeed Studio ecosystem, making it easier to integrate the display, controller, audio system, and supporting peripherals.

• 4.26-inch E-Paper Display — 800 × 480: E-paper was selected instead of a conventional LCD because CraveGuide primarily displays static information such as recommendations, menu items, prices, and order status. E-paper consumes very little power while maintaining an image, making it particularly suitable for a battery-powered portable device. It also provides excellent readability under ambient light, giving the device a more paper-like appearance that fits naturally into a restaurant environment.
• Seeed Studio EE04 E-Paper Display Driver: The display is controlled using the Seeed Studio XIAO ePaper Display Board EE04, which integrates a XIAO ESP32-S3 Plus and supports both 24-pin and 50-pin e-paper displays, including the 4.26-inch 800 × 480 panel used in this project. One of the major benefits of EE04 is that it combines several functions required by a portable e-paper device into a single board. It provides a JST battery connector, built-in battery charging circuitry, battery power switch, and three user-programmable buttons, reducing the amount of additional power-management and interface hardware required.
• Seeed Studio reSpeaker Flex — 4-Mic Linear Array: For voice input, CraveGuide uses the Seeed Studio reSpeaker Flex Linear 4-Mic Array. The four microphones are arranged in a linear configuration, making it well suited for a front-facing device such as a restaurant ordering terminal. The linear array provides directional voice pickup toward the user while helping suppress unwanted sound from the rear. A major advantage of the reSpeaker Flex is that it is much more than four microphones. Its XMOS XVF3800 audio processor provides dedicated audio-processing capabilities including beamforming, Voice Activity Detection (VAD), Acoustic Echo Cancellation (AEC), noise suppression, direction-of-arrival processing, and de-reverberation.
• Li-Ion Battery: A rechargeable Li-Ion cell provides portable power to the device. Battery operation allows CraveGuide to be placed on a restaurant table without requiring a continuous external power connection. The EE04's integrated battery interface and charging circuitry simplify the power architecture by allowing the rechargeable cell to be connected directly through its battery connector.
• 5 V Boost Converter: A 5 V boost converter is used where a regulated 5 V supply is required by the connected hardware. It allows the lower battery voltage to be converted into a suitable higher supply rail while maintaining the portable nature of the system.
• Speaker: A speaker is included for future audio feedback and voice interaction. This can allow CraveGuide to provide spoken responses such as confirmation messages, recommendations, or order-status information, extending the system from a voice-input device toward a complete voice-interactive terminal.

Custom Enclosure Design and Component Assembly

To make CraveGuide suitable for practical use in a restaurant environment, a custom enclosure was designed to house the electronic components while maintaining a compact and clean appearance. The enclosure was designed around the dimensions of the 4.26-inch e-paper display and the internal electronics, with the display positioned on the top surface for easy viewing by the user.

The microphone openings are positioned toward the user to provide clear voice capture, while the speaker and power switch are placed to remain accessible without interfering with normal operation. The physical buttons are positioned on the side.The components were assembled in a modular manner, with the display/controller and microphone system acting as the two primary modules. The modules are connected through UART communication, while the battery and power circuitry provide the required supply rails.

The final enclosure transforms the individual development boards and modules into a single integrated tabletop device.

Connection Diagram

The two modules communicate through a dedicated UART serial link, allowing the display module to send commands and receive the resulting voice transcription.

For the UART connection, the ESP32-S3 uses a hardware serial interface with separate TX and RX lines. The connections are made in a crossed configuration: the TX of one module is connected to the RX of the other, and vice versa. A common ground is shared between both modules.

Mic Module UART_RX D2 -- Display Module UART_TX D15

Mic Module UART_TX D3 -- Display Module UART_RX D14

System Architecture

CraveGuide follows a two-module embedded architecture in which voice processing and user-interface functions are separated between two ESP32-S3-based modules. This separation keeps the system modular and allows the microphone subsystem to handle audio-specific processing while the display subsystem focuses on the application logic and user interaction.

Microphone and Voice Processing Module: The microphone module is built around the Seeed Studio reSpeaker Flex Linear 4-Mic Array, which uses the XMOS XVF3800 audio processor. The microphone array captures the user's voice, while the XVF3800 performs dedicated audio processing such as beamforming, voice activity detection, acoustic echo cancellation, noise suppression, automatic gain control, direction-of-arrival processing, and dereverberation.

For embedded operation, the reSpeaker Flex is configured in I²S mode. In this configuration, the processed audio from the XVF3800 is transferred digitally to the host ESP32-S3 through the I²S interface. Seeed Studio specifically provides I²S firmware for using the XVF3800 with embedded hosts such as the XIAO ESP32-S3.

The microphone module is responsible for:

• Receiving the recording command from the display module.
• Capturing approximately 3 seconds of voice input.
• Obtaining processed audio from the XVF3800 through I²S.
• Preparing the recorded audio for transcription.
• Sending the audio to the ElevenLabs Speech-to-Text API.
• Receiving the transcribed text.
• Sending the final text back to the display module through UART.

ElevenLabs provides a Speech-to-Text API that accepts an audio file and returns the corresponding transcription; the current API documentation identifies Scribe v2 as its STT model.

Display and Application Module

The second module is the Seeed Studio EE04 e-paper display board, based on the XIAO ESP32-S3. It controls the 4.26-inch 800 × 480 e-paper display and acts as the main application controller. The EE04 supports the 4.26-inch monochrome 800 × 480 panel used in CraveGuide.

The display module is responsible for:

• Displaying the user-interface screens.
• Detecting the user's button interactions.
• Sending the recording command to the microphone module.
• Receiving the final transcription through UART.
• Extracting taste and dietary keywords.
• Searching the local menu database.

ElevenLabs Speech-to-Text

ElevenLabs acts as the cloud-based transcription layer of the system. The recorded audio is uploaded to the Speech-to-Text service, which returns the recognized text. CraveGuide then uses this text as the input to its local, rule-based recommendation system.

Importantly, ElevenLabs is being used only for Speech-to-Text in the current implementation. The actual food recommendation is performed locally using the predefined menu database and keyword-based filtering rather than an AI recommendation model.

Code Implementation

The firmware for CraveGuide was developed using the Arduino IDE. The application is divided into two independent firmware programs: one running on the microphone module and the other running on the display/application module.

Microphone Module Firmware

The microphone module is responsible for the complete audio-acquisition and Speech-to-Text pipeline. It interfaces with the reSpeaker Flex 4-Mic Array and communicates with the onboard XMOS XVF3800 through the I²S interface. The processed digital audio is captured by the ESP32-S3, stored temporarily, and then sent to the ElevenLabs Speech-to-Text service.

Development Environment

• IDE: Arduino IDE
• Microcontroller: ESP32-S3
• Microphone: Seeed Studio reSpeaker Flex Linear 4-Mic Array
• Audio interface: I²S
• Audio processor: XMOS XVF3800
• Communication with display: UART
• Speech-to-Text: ElevenLabs API

We are using AudioTools Arduino Library for I2S communication with XVF3800.

The main loop() continuously monitors the UART interface. When the display module sends the record command (r), the microphone module starts the recording process.

The main functions are separated according to their individual responsibilities.

recordAudio() handles the recording process. It receives the processed audio through I²S and stores the captured PCM samples in memory for subsequent processing.

sendToElevenLabs() handles the Speech-to-Text stage. The recorded audio is prepared and transmitted to the ElevenLabs API, which returns the recognized speech as text.

UART communication is used to coordinate the microphone and display modules. The display sends the recording command, while the microphone module returns the final transcription.

Future Scope

The current implementation provides a strong foundation that can be extended into a more intelligent and complete restaurant-assistance system. Future improvements could include:

• AI-based recommendation: Replace the current keyword-based filtering with an AI/NLP system capable of understanding more natural and complex food preferences.
• Larger and dynamic menu database: Connect the device to a restaurant's live menu so that dishes, prices, availability, and offers can be updated automatically.
• Multi-language voice support: Allow customers to speak in regional languages and mixed-language sentences.
• Offline voice processing: Add local Speech-to-Text capabilities to reduce dependence on Internet connectivity.
• Order management integration: Connect CraveGuide with the restaurant's kitchen or Point-of-Sale system so confirmed orders can be transmitted automatically.
• Order status tracking: Display real-time states such as Order Received → Preparing → Ready → Served.
• Audio feedback: Use the speaker to provide spoken recommendations and order confirmations.

Conclusion

CraveGuide demonstrates a practical approach to making the food-selection and ordering process more natural, simple, and interactive. Instead of requiring users to browse through a large menu, the system allows them to express their cravings through voice. The reSpeaker Flex microphone array and XMOS XVF3800 handle the audio acquisition and processing, while ElevenLabs Speech-to-Text converts the recorded speech into text. The display module then processes the transcription, identifies relevant taste and dietary preferences, and searches the predefined menu database to provide suitable recommendations.

Overall, the project successfully combines embedded hardware, voice interaction, cloud-based Speech-to-Text, and a rule-based recommendation system into a functional tabletop food-ordering prototype.