AI-driven LoRa & LLM-enabled Kiosk & Food Delivery System

After scrutinizing pioneering research papers on AI-powered applications and improvements for restaurant food preparation and delivery, I had become fascinated by the prospects of AI-oriented solutions in the food service industry. Thus, I have started to conceptualize a state-of-the-art restaurant from the ground up to not only employ AI-assisted solutions to enhance a preexisting restaurant workstation in limited aspects to automate food preparation and service procedures, as extensively covered in the aforementioned research papers, but also utilize AI-based algorithms to give customers a considerable degree of autonomy in generating user-specific menus/deals based on their preferences to provide an authentic and personalized customer experience. In other words, I focused on developing a restaurant establishment from scratch, providing AI-assisted features in customer relations, special menu/deal generation, kiosk/web dashboard interactions, order management, food preparation, and food service processes.

While conceptualizing all of the AI-assisted features I wanted to implement in my hypothetical restaurant installation, I conducted extensive research about various restaurant types to pinpoint the best establishment layout that would effectively showcase my concepts and solutions. In this regard, I decided to base my restaurant establishment layout on popular drive-through restaurants since their fast-paced service requirements and high customer retention rates provide the ideal conditions to examine and emphasize my AI-assisted solutions as a proof-of-concept research project.

Considering a drive-through restaurant's structure and requirements, I started to work on determining my objectives regarding my AI-assisted solutions that would improve customers' overall impression by providing a personalized, attentive, and consistent experience from the restaurant web application (and dashboard) to the kiosk customer endpoint.

After considering different networking options between the restaurant web application and the kiosk customer endpoint, I decided to utilize LoRaWAN due to its long-range coverage, low power consumption, and consistency, especially for handling simultaneous and interconnected operations of a fast-paced drive-through restaurant.

As opposed to the usual drive-through restaurant customer experience, in accordance with my AI-powered solutions, I concentrated on providing customers with the autonomy to generate user-specific menus/deals based on their preferences by employing different large language models (LLMs) enabled by the restaurant web application. In this regard, the web application allows the selected LLM to access customer preferences, available food item information (name, price, etc.), and food categories from the database to generate user-specific menus/deals. While producing menus/deals, the selected LLM determines the menu theme, description, the offered food item list, and the applied discount percentage.

In addition to the LLM-generated user-specific menus/deals, I decided to develop AI-assisted features to recognize registered customer vehicles for account authorization and identify food prep stations for performing the automatic food delivery process precisely in order to provide an outstanding AI-oriented customer experience. Nonetheless, I chose not to implement an involuntary data collection process for customer vehicles since I did not want to build a 1984-esque drive-through restaurant establishment :) In this regard, I developed vehicle image collection and account authorization based on vehicle recognition as opt-in restaurant features.

So, my initial objectives became as follows.

🤖 Objectives

✅ Developing a full-fledged web application to enable customers to create user accounts and provide contact information, payment settings, and menu/deal preferences.

✅ Enabling the web application to generate unique 4-digit authentication keys for each customer account.

✅ Preparing the available food item list and generic menus/deals offered by the drive-through restaurant.

✅ Utilizing the web application as the main drive-through kiosk interface to present generic menus/deals that would be available to customers with or without a user account.

✅ Enabling the web application to employ different LLMs to give customers the autonomy to generate user-specific menus/deals based on their preferences, distinguished by unique order tags.

✅ Providing restaurant workers with a real-time order schedule via the web application, including thorough menu information and the requested items per food prep station.

✅ Developing the kiosk customer endpoint, which would allow customers to order generic or user-specific menus/deals, collect image samples of their vehicles, and authorize their user accounts on the web application via vehicle detection (opt-in).

✅ Enabling the kiosk customer endpoint to communicate with the restaurant web application via LoRaWAN.

✅ Building a web-enabled food delivery mechanism to provide customers with the requested food items automatically, which would communicate with the web application to obtain the latest placed order information and update the order status.

In accordance with my initial objectives, I started to develop my stated AI-assisted drive-through restaurant features and mechanisms.

First, I decided to employ Ollama to enable the restaurant web application to capitalize on various LLMs (large language models) locally without any third-party service or paywall. Since I had been utilizing LattePanda Mu (N305) with its full-function evaluation carrier board, which is a powerful and capable SBC (single-board computer), I was able to generate reliable responses with relatively high speeds by using solely the onboard CPU — octa-core.

Then, I started to develop the kiosk customer endpoint, which handles customer-to-kiosk interactions and AI-based vehicle identification. As discussed, I decided to utilize LoRaWAN to establish the workflow between the endpoint and the restaurant web application. Thus, I based the customer endpoint design on the budget-friendly and feature-rich RA-08H LoRaWAN node board. Of course, the kiosk customer endpoint would not be complete without a dedicated LoRaWAN gateway transferring data packets to the web application with optimized configurations. Thus, I decided to utilize the LR1302 LoRaWAN gateway module and its Raspberry Pi-compatible hat. Also, I designed a unique PCB for the kiosk customer endpoint to build a compact and user-friendly device.

In order to process the LoRa-transmitted data packets from the customer endpoint, I connected the LoRaWAN gateway to The Things Network. Then, I enabled the restaurant web application to run a PHP-based MQTT client to access the LoRa-transmitted data packets continuously through The Things Stack MQTT server.

Finally, I started to develop the automatic food delivery system, which communicates with the web application to obtain the latest order information and collects the requested food items from the food prep stations to serve them to customers. After considering various design approaches, I decided to base my food delivery system on the H-Bot gantry mechanism, which is driven by a single GT2 timing belt and provides a high level of precision for simple sorting and conveyor transfer operations.

Since I wanted to make the food delivery system capable of identifying individual food prep stations via object detection, I decided to design unique AprilTag signs for each prep station. Thus, I decided to utilize Arduino Nicla Vision as the processor of my food delivery system, which is an easy-to-use and scalable development board with a built-in camera. Since I designed an H-Bot-inspired mechanism, a regular (Rigid) PCB would not allow me to develop the features I envisioned for the gantry head serving the collected food items. Hence, I decided to design a unique Flex PCB for the food delivery system, which would include stiffeners to enhance mechanical strength.

So, my preliminary tasks to complete this research project became as follows.

🤖 Preliminary Tasks

✅ Establishing Ollama on LattePanda Mu and installing distinct open-source LLMs (large language models).

✅ Optimizing, fine-tuning, and testing the available LLMs to pinpoint the most suitable ones for user-specific menu/deal generation.

✅ Establishing the LoRaWAN data transfer procedure between the kiosk customer endpoint and The Things Network via the dedicated LoRaWAN gateway.

✅ Installing the PHP-MQTT client on LattePanda Mu to enable the restaurant web application to obtain LoRa-transmitted data packets from The Things Network.

✅ Enabling the web application to present the latest LoRa transmission logs and the current order status automatically.

✅ Prototyping and designing the kiosk customer endpoint PCB.

✅ Designing assembly parts for the kiosk customer endpoint and the vehicle platform.

✅ Designing assembly parts and AprilTag signs for the food prep stations.

✅ Prototyping and designing the food delivery system Flex PCB with stiffeners.

✅ Designing assembly and mechanical parts for the automatic food delivery system inspired by the H-Bot kinematic structure.

✅ Building a FOMO object detection model with Edge Impulse for recognizing the registered customer vehicles.

✅ Building a FOMO object detection model with Edge Impulse for identifying individual food prep stations by their assigned AprilTag signs.

After completing the mentioned tasks and rigorously examining various LLMs for user-specific menu/deal generation, I decided to enable these models on the restaurant web application to provide a wide range of options:

• deepseek-r1:8b
• deepseek-r1:7b
• deepseek-r1:1.5b
• gemma3:4b
• gemma3:1b
• llama3.2:3b
• qwen3:4b
• phi4-mini

After concluding all design, networking, building, and programming steps that I could not cover in this already long introduction, I finalized my proof-of-concept research project, showcasing a full-fledged drive-through restaurant enhanced with the AI-assisted features I envisioned.

By referring to the following tutorial, you can inspect the in-depth feature, design, and code explanations with the challenges I faced during the overall development process.

🎁📢 Huge thanks to ELECROW for sponsoring this project by providing their high-quality PCB manufacturing services and feature-rich products:

⭐ Elecrow Regular PCB Service

⭐ Elecrow Premium Flex PCB Service

⭐ LR1302 LoRaWAN Long Range Gateway Module SPI (EU868 SX1302) | Inspect

⭐ LR1302 868M/915M LoRaWAN Hat for RPI (SX1302) | Inspect

⭐ RA-08H LoRaWan Node Board with RP2040 | Inspect

⭐ Crowtail - Serial Camera | Inspect

🎁📢 Huge thanks to DFRobot for sponsoring this project by providing their high-performance and feature-rich products:

⭐ LattePanda Mu (N305 CPU, 16GB RAM, 64GB eMMC) | Inspect

⭐ LattePanda Mu Full-Function Evaluation Carrier Board | Inspect

⭐ LattePanda Mu Aluminum Active Cooler | Inspect

⭐ Fermion: 2.0" IPS TFT LCD Display (320x240) | Inspect

Design process, interconnected mechanisms, available features, and final results

To effectively showcase this complex research project on developing AI-based drive-through restaurant features, I decided to create a meticulously written tutorial and produce comprehensive demonstration videos that include my entire development process, experiments, and results from start to finish.

In this regard, I highly recommend inspecting the complementary parts of the project demonstration videos while reading the written tutorial.

Step 1: Setting up RA-08H LoRaWAN node board on Arduino IDE

RA-08H LoRaWAN node board comes with an integrated Raspberry Pi RP2040 and supports frequency bands between 803MHz and 930MHz for the long-range (LoRa) data transmission. It also provides six Crowtail interfaces for different sensors and a 20-pin header for further development requirements.

#️⃣ To enable the recommended RP2040 core, go to File ➡ Preferences ➡ Additional boards manager URLs and add the given board package URL:

https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json

#️⃣ To install the enabled board core, navigate to Tools ➡ Board ➡ Boards Manager, search for RP2040, and select Raspberry Pi Pico/RP2040.

#️⃣ Even though the 2.6.0 version is officially recommended by the RA-08H node board documentation, I encountered some compatibility issues while trying it on Arduino IDE 2.0+. After upgrading the board core to the latest version at the time (4.5.1), I did not encounter any errors.

#️⃣ After installing the core successfully, select Tools ➡ Board ➡ Raspberry Pi Pico/RP2040 ➡ Raspberry Pi Pico. Then, select the associated COM port.

#️⃣ If the RA-08H node board just shows a default COM port number (e.g., COM3) without any board information, upload a basic example such as Blink.

#️⃣ If Arduino IDE does not show the COM port for RP2040 due to faulty code or connection issues, hold down the RP2040 BOOT button and press the RESET button to enable the direct firmware upload mode.

#️⃣ After configuring the RA-08H LoRaWAN node board on Arduino IDE, I installed the libraries required to control the attached electronic components:

📚 Keypad (built-in) | Inspect

📚 DFRobot_GDL | Download

📚 Adafruit_VC0706 (Serial Camera Library) | Download

❗ During my initial experiments with the electronic components, I realized that RP2040 could not accurately register the characters/symbols of a custom keypad map created via the Keypad library. After troubleshooting, I found out that the pullup resistor configurations of the Keypad library are not compatible with RP2040. Thus, I modified the Keypad.h file to make the INPUT_PULLUP configuration as default, as shown via the highlighted lines below. After fixing the resistor configuration issue, I did not encounter any problems with my custom keypad.

Documents\Arduino\libraries\Keypad\src\Keypad.h

Step 2: Setting up Arduino Nicla Vision on Arduino IDE

Nicla Vision is a compact but powerful development board providing a GC2145 2MP color camera, a 6-axis motion sensor, a VL53L1X Time of Flight (ToF) distance sensor, and an omnidirectional microphone.

#️⃣ To install the required core for Nicla boards, navigate to Tools ➡ Board ➡ Boards Manager and search for Mbed OS Nicla Boards.

#️⃣ After installing the core successfully, navigate to Tools ➡ Board ➡ Arduino Mbed OS Nicla Boards and select Arduino Nicla Vision.

#️⃣ If Nicla Vision throws an error while attempting to utilize the built-in Wi-Fi module for the first time, you need to install the Wi-Fi firmware manually. Since I bought Nicla Vision more than a year ago, I am not sure if this is still an issue with the newer batches. If so, to install the Wi-Fi firmware manually, go to Examples ➡ STM32H747_System ➡ WiFiFirmwareUpdater and execute the provided code.

#️⃣ After running the WiFiFirmwareUpdater program, Arduino IDE flashes Nicla Vision automatically to install the required Wi-Fi firmware and certificates.

#️⃣ After configuring Nicla Vision on Arduino IDE, I installed the library required to control the onboard Time of Flight (ToF) sensor via the library manager. To install the VL53L1X library, open the built-in Library Manager and search for VL53L1X, authored by Pololu.

Step 3: Setting up LR1302 LoRaWAN gateway on Raspberry Pi 4 Model B

LR1302 is a compact and versatile LoRaWAN gateway module based on Semtech Network's SX1302 LoRaWAN® baseband chip (concentrator), providing reliable long-distance wireless transmission. Since LR1302 utilizes the Mini-PCIe form factor design, it is possible to use LR1302 with different gateway boards.

The official LR1302 expansion board is a Raspberry Pi-compatible hat and includes an onboard GPS module and RTC (real-time clock) to improve the performance and accuracy of LR1302. Since the expansion board comes with a built-in cooling fan, it reduces Raspberry Pi's performance issues due to insufficient heat dissipation.

Since I had a spare Raspberry Pi 4 Model B, I employed it to run my dedicated LR1302 LoRaWAN gateway. Nonetheless, the LR1302 expansion board supports up to Raspberry Pi 5.

#️⃣ First, I installed the official Raspberry Pi 4 B-compatible operating system image on a microSD card to start developing with a fresh OS.

#️⃣ After setting up Raspberry Pi OS successfully, I attached the LR1302 gateway module to its official Raspberry Pi-compatible hat via the Mini-PCIe slot connector. Then, I established the U.FL (IPEX) antenna interface connection between LR1302 and the expansion board.

#️⃣ I attached the LR1302 expansion board to Raspberry Pi via the 40-pin GPIO connector. Then, I connected the expansion board's GPS+BD dual-mode antenna and long-range (LoRa) transmission antenna — 868MHz/915MHz — via the integrated SMA connectors.

#️⃣ Since the LR1302 gateway module communicates with Raspberry Pi via SPI and I2C protocols, which are not enabled by default, I activated SPI and I2C interfaces via the built-in configuration tool. To access the configuration tool, go to Preferences ➡ Raspberry Pi Configuration.

#️⃣ Then, I rebooted my Raspberry Pi to apply interface changes.

❗ The serial console cannot be utilized to access Raspberry Pi since the GPS module of the expansion board uses the hardware UART pins.

#️⃣ After enabling the required interfaces, I downloaded the official SX1302 concentrator firmware files directly from GitHub.

#️⃣ After extracting the downloaded concentrator firmware files in the root (home) folder, I opened the sx1302_hal folder on the terminal.

#️⃣ Then, to compile the installed SX1302 concentrator firmware and create its final executable on my Raspberry Pi, I executed the make command in the opened sx1302_hal folder.

make

Step 4: Creating a user account on The Things Network and registering LoRaWAN gateway information

The Things Network (TTN) is a well-known collaborative IoT development ecosystem that provides feature-rich tools and an open network to create scalable solutions and devices employing LoRaWAN. Furthermore, The Things Network enables an open LoRaWAN network server, The Things Stack, which is the backbone of the LoRaWAN development process with TTN since the network server allows developers to securely manage applications, end devices, and gateways.

#️⃣ First, I navigated to the TTN console login page and created my The Things ID — TTN user account.

eu1.cloud.thethings.network/oauth/login

#️⃣ After confirming my email address to enable my Things ID, I selected the recommended Australia (au1) LoRaWAN network cluster based on my location — Turkey.

#️⃣ Then, I was able to access my TTN console built on The Things Stack (Sandbox).

#️⃣ To register my dedicated LR1302 LoRaWAN gateway, I navigated to Dashboard ➡ Register gateway.

#️⃣ The Things Network requires a specific 64-bit gateway EUI (Extended Unique Identifier) to establish a connection with a LoRaWAN gateway. Since the LR1302 LoRaWAN gateway module does not include a permanent EUI assigned by the manufacturer to provide versatile development options, I decided to utilize an online EUI generator to obtain my specific gateway EUI.

❗ If you have a different gateway module with a factory-default EUI, you must use the assigned module EUI to register your gateway.

a6ac5bb7ada286be

A6AC5BB7ADA286BE

0xA6, 0xAC, 0x5B, 0xB7, 0xAD, 0xA2, 0x86, 0xBE

#️⃣ After obtaining my specific gateway EUI, I entered the required gateway information and chose the associated frequency plan with my LR1302 gateway module — EU868 (863-870 MHz).

#️⃣ If you are using the US915 (902-928 MHz) version of the LR1302 gateway module, you must choose the frequency plan accordingly.

• Gateway ID: "kiosk-customer-info1s"
• Gateway name: "Kiosk Customer Info"
• Frequency plan: "Europe 863-870 MHz (SF9 for RX2)"

#️⃣ After successfully registering the LR1302 gateway, I needed to assign the gateway server address in order to route the LoRa-transmitted data packets through the selected LoRaWAN network cluster. In my case, it is the Australia (au1) cluster. Nevertheless, you should assign the gateway server address according to your recommended cluster — eu1, nam1, or au1.

#️⃣ To access the gateway settings, go to Gateway name [Kiosk Customer Info] ➡ General settings.

au1.cloud.thethings.network

Step 4.1: Configuring the LR1302 LoRaWAN gateway module according to the registered gateway information on TTN

As mentioned earlier, the LR1302 LoRaWAN gateway module lets developers modify the gateway ID and network configurations to provide different deployment options.

Thus, after completing setting up my dedicated LR1302 LoRaWAN gateway on The Things Network console, I needed to modify the gateway configurations on Raspberry Pi 4 B.

#️⃣ First, I navigated to the sx1302_hal ➡ packet_forwarder folder and also opened it on the terminal.

#️⃣ According to my LR1302 gateway module version — EU868 — and the transmission protocol employed by Raspberry Pi — SPI, I found the accurate config file to modify gateway settings accurately.

global_conf.json.sx1250.EU868

#️⃣ After finding the correct configuration file, I edited it via nano (command-line text editor) on the terminal to modify gateway settings according to the registered gateway information on The Things Network.

nano global_conf.json.sx1250.EU868

#️⃣ To establish the connection between my dedicated LR1302 gateway and The Things Network, I needed to change the gateway_ID and server_address variables under gateway_conf. If the serv_port_up and serv_port_down variables are not set to 1700 by default, you must also change them to 1700.

"gateway_conf": {
	"gateway_ID": "A6AC5BB7ADA286BE",
	/* change with default server address/ports */
	"server_address": "au1.cloud.thethings.network",
	"serv_port_up": 1700,
	"serv_port_down": 1700,
	
	...

#️⃣ To save and close files on nano, press CTRL + x, then Y for the prompt.

Step 4.2: Initiating the LR1302 LoRaWAN gateway with the modified configurations for TTN

#️⃣ After modifying the associated configuration file successfully, I executed the integrated LoRa packet forwarder — lora_pkt_fwd — with the configured gateway settings under the sx1302_hal ➡ packet_forwarder folder via the terminal in order to initiate the data transfer process between my dedicated LR1302 gateway and The Things Network.

./lora_pkt_fwd -c global_conf.json.sx1250.EU868

#️⃣ To run the LoRa packet forwarder — lora_pkt_fwd — successfully, the reset_lgw.sh file must be under the same location — sx1302_hal ➡ packet_forwarder. Otherwise, the packet forwarder cannot reset the SX1302 baseband chip.

#️⃣ After running the packet forwarder for the first time, I encountered some file permission errors on my Raspberry Pi 4 B. Thus, I executed the command line below on the terminal to make the SX1302 concentrator firmware files executable by changing the parent folder permissions.

sudo chmod -R 777 /home/kutluhan_pi_lora/LR1302_LoRa

#️⃣ After changing the file permissions, I was able to run the integrated LoRa packet forwarder without a problem.

#️⃣ Then, I checked my The Things Network console to review the connection status of my dedicated LR1302 LoRaWAN gateway.

#️⃣ To stop the LoRa packet forwarder and the established TTN connection, press CTRL + c on the terminal. After stopping the packet forwarder, the TTN console updates the LR1302 gateway connection status to disconnected immediately.

Step 5: Creating an application on The Things Network console to link the RA-08H LoRaWAN node board to the LR1302 LoRaWAN gateway

After adding my dedicated LR1302 LoRaWAN gateway to my LoRaWAN network server provided by The Things Stack, I needed to link my RA-08H LoRaWAN node to my registered gateway by creating an application in my TTN console.

Essentially, a TTN application allows developers to add end devices (nodes) to a registered LoRaWAN gateway. Then, the application processes the data packets received from the registered gateway to verify whether they are from a linked LoRaWAN node or not. If so, the application logs the received data packets under the linked node.

Once a LoRaWAN node is added to a registered LoRaWAN gateway, the gateway and the node utilize the Over-the-Air-Activation (OTAA) activation method to establish data transmission.

#️⃣ First, I navigated to Dashboard ➡ Register gateway and entered the required application information.

Application ID: kiosk-customer-end1

Application name: Kiosk Customer End

#️⃣ After creating my TTN application successfully, I navigated to Application name [Kiosk Customer End] ➡ End devices to register my RA-08H LoRaWAN node manually.

#️⃣ Then, I selected the frequency plan and the LoRaWAN version, corresponding to the registered LR1302 LoRaWAN gateway information.

Frequency plan: Europe 863-870 MHz (SF9 for RX2)

LoRaWAN version: LoRaWAN Specification 1.0.2

Regional Parameters version: RP001 Regional Parameters 1.0.2

#️⃣ Similar to the LR1302 LoRaWAN gateway, the RA-08H LoRaWAN node board is programmable and does not include permanent EUIs or keys assigned by the manufacturer to provide versatile development options. Thus, I was able to utilize the built-in EUI and key generation tools provided by the TTN console.

❗ If you have a different LoRaWAN node with factory-default EUI configurations, you must use the assigned information to register your node.

#️⃣ To register a unique end device node, TTN requires JoinEUI (formerly called AppEUI), DevEUI, and AppKey. I assigned a generic JoinEUI and employed the built-in generation tools to assign the remaining variables.

#️⃣ After assigning the required EUIs and key, I entered a unique end device ID to complete the LoRaWAN node registration.

JoinEUI: 1111111111111111

DevEUI: 70B3D57ED006FD87

AppKey: 93C4BC655918ADE35ABA104776C41FD0

End device ID: kiosk-customer-end-device1

Step 5.1: Transmitting data packets from the RA-08H LoRaWAN node board to The Things Network via the LR1302 LoRaWAN gateway

After creating my TTN application and registering the required end device (node) information on my TTN console, I needed to configure my RA-08H LoRaWAN node board according to the registered end device provisioning information for Over-The-Air-Activation (OTAA).

Since the integrated RP2040 can communicate with the RA-08H module via serial communication (hardware), the RA-08H node board enables developers to modify node settings via AT commands effortlessly.

In this regard, I was able to create functions to configure (reset) RA-08H settings automatically and transmit data packets via the configured RA-08H module. Since I programmed these functions for initial testing and debugging, I revised them until concluding the kiosk customer endpoint features. So, please refer to the following steps or the ai_driven_customer_service_kiosk_lora_end_point.ino file to review the final versions.

As explained earlier, the LoRa-transmitted data packets are transferred to the LoRaWAN network server provided by The Things Stack through the linked LR1302 LoRaWAN gateway.

#️⃣ First, I programmed a function to modify RA-08H settings via AT commands according to the registered TTN end device (node) information.

// Set the node access method to OTAA
  Serial1.print("AT+CJOINMODE=0\n"); 

  // Set the node group frequency mask (Set channels 0-7)
  Serial1.print("AT+CFREQBANDMASK=0002\n"); 

  // Set the node type
  Serial1.print("AT+CCLASS=2\n");

  // Assign the associated TTN application end device DEVEUI
  Serial1.print("AT+CDEVEUI=" + DEVEUI + "\n");

  // Assign the associated TTN application end device JOINEUI (formerly called APPEUI)
  Serial1.print("AT+CAPPEUI=" + APPEUI + "\n");

  // Assign the associated TTN application end device APPKEY
  Serial1.print("AT+CAPPKEY=" + APPKEY + "\n"); 

  // Set the node uplink and downlink frequency — 1 for same frequency, 2 for different frequency 
  Serial1.print("AT+CULDLMODE=2\n");  

  // Set the node to join the network
  Serial1.print("AT+CJOIN=1,1,10,3\n");

#️⃣ After configuring the RA-08H module settings, I opened my TTN console to check the connection status between the dedicated LR1302 LoRaWAN gateway and the RA-08H node board, established through the LoRaWAN network server.

#️⃣ As I saw that the RA-08H node board was active and verified on my LoRaWAN network server provided by The Things Stack, I programmed another function to review the specific data packet transfer success, such as 1111 and 1213.

// Send a specific data packet to the linked LR1302 LoRaWAN gateway module.
  Serial1.print("AT+DTRX=1,2,3," + data_packet + "\n");

Step 5.2: Configuring The Things Stack MQTT server integration settings

Even though The Things Network (TTN) provides various integration methods to manage streaming events and receive updates from registered LoRaWAN nodes (end devices), I decided to utilize The Things Stack MQTT server to obtain the latest LoRa-transmitted data packets from the RA-08H node board since I wanted to develop a stable and reliable restaurant web application as the main drive-through kiosk interface, considering it must process commands from the kiosk customer endpoint without any mistakes or interruptions.

MQTT is a lightweight publish-and-subscribe messaging protocol designed for machine-to-machine connections with resource constraints or limited network bandwidths, which is substantial for building dependable IoT (Internet of Things) applications. Considering MQTT is a bi-directional communication protocol and provides a lossless connection with no queues, it is the ideal integration method to obtain information from a LoRaWAN network, which can support a vast number of nodes (end devices).

As The Things Stack (TTS) exposes an MQTT server and creates an MQTT endpoint for each TTN application, the TTN console provides all of the necessary information to connect MQTT clients so as to subscribe to uplink messages (traffic) or publish downlink messages (traffic). Although the TTS MQTT server supports popular third-party MQTT clients, I decided to enable my restaurant web application to run a PHP-based MQTT client locally to minimize the network load as much as possible. Please refer to the following steps to get more information about programming a PHP-based MQTT client from scratch.

#️⃣ First, I navigated to Application name [Kiosk Customer End] ➡ Other integrations ➡ MQTT.

#️⃣ Then, I started to collect the provided The Things Stack MQTT server information, required to connect my MQTT client.

#️⃣ TTS MQTT server host addresses are in accordance with the application's LoRaWAN network cluster — au1.

Public address: au1.cloud.thethings.network:1883

Public TLS address: au1.cloud.thethings.network:8883

#️⃣ To establish a secure connection, TTS provides unique credentials for each application. It assigns the MQTT username based on the application ID and the tenant ID — ttn.

{application id}@{tenant id}

#️⃣ Since TTS does not assign an MQTT password by default, called the API key, I generated a new API key for MQTT client authentication. After generating the MQTT password, I saved the produced API key since API keys disappear once the TTN console is closed due to security reasons.

#️⃣ After obtaining the required MQTT server information, I also noted the registered end device ID of the RA-08H LoRaWAN node board since TTS MQTT server exposes MQTT topics by employing unique end device IDs.

MQTT Username: kiosk-customer-end1@ttn

MQTT Password: NNSXS.TCN35UJOVNRK65RGYLKC6AZYA3AHQXVOCIOZ6MA.JMB4PR7EWW6RZ7JO32CK4N7FDAS6KX766TA6ZI6W3FTQ65VEB7KQ

End device ID: kiosk-customer-end-device1

MQTT topics are UTF-8 strings utilized by the MQTT protocol to identify and route messages — uplink and downlink traffic — between MQTT publishers and subscribers.

• v3/{application id}@{tenant id}/devices/{device id}/join
• v3/{application id}@{tenant id}/devices/{device id}/up
• v3/{application id}@{tenant id}/devices/{device id}/down/sent
• v3/{application id}@{tenant id}/devices/{device id}/down/push

Step 6: Setting up LattePanda Mu with the Full-Function Evaluation carrier board

LattePanda Mu is a powerful and versatile x86 compute module featuring Intel i3-N305 octa-core processor, 16GB LPDDR5 memory, and 64GB eMMC storage. Since LattePanda provides feature-rich peripherals for the Mu family and supports multiple operating systems, it is effortless to utilize LattePanda Mu as a scalable development platform.

To utilize Lattepanda Mu as a feature-rich SBC (single-board computer) by exposing its pins, I decided to employ the full-function evaluation carrier board since I needed to access the full potential of Lattepanda Mu in order to develop a full-fledged restaurant web application as the main drive-through kiosk and customer interface.

#️⃣ First, I connected the heatsink (active aluminum cooler) to the Lattepanda Mu compute module.

#️⃣ Then, I attached the Lattepanda Mu compute module to the evaluation carrier board and connected the cooling fan.

#️⃣ To enable Lattepanda Mu to establish Wi-Fi connections, I decided to attach an AC8265 wireless dual-mode NIC (2230) via the M.2 E Key of the evaluation board.

#️⃣ However, the M.2 E Key is multiplexed with the second 2.5GbE NIC and disabled by default. So, I needed to switch the associated jumper to M2_WIFI to activate the M.2 E Key.

#️⃣ Since I needed to run large language models locally on Lattepanda Mu without connecting a GPU, the onboard 64GB eMMC would engender some instability issues while executing CPU-only LLM operations, running an MQTT server, and hosting the restaurant web application simultaneously. Thus, I decided to utilize a SanDisk Ultra 500GB SSD as the storage.

#️⃣ Nonetheless, the evaluation board does not contain an M.2 M Key slot for M.2 NVMe SSDs. Thus, I needed to use the official PCIe NVMe adaptor to connect my 500GB SSD.

#️⃣ Thankfully, the evaluation board provides a DC power screw terminal plug (5.08 mm). So, I was able to modify my compatible 19V power supply salvaged from my previous laptop to connect it directly to the evaluation board.

#️⃣ After concluding the full-function evaluation carrier board hardware assembly, I set up Ubuntu as the operating system for LattePanda Mu by creating the installation media for Ubuntu 24.04.2, which is officially supported by LattePanda, via Rufus. You can inspect the prioritized operating system versions here.

Step 7: Setting up LAMPP web server on LattePanda Mu

As mentioned earlier, I wanted to run all restaurant web application operations locally on LattePanda Mu. Thus, I decided to utilize XAMPP as the local server for my web application, providing all necessary tools to build a full-fledged kiosk and customer interface.

Plausibly, the XAMPP application provides an official Linux installer — LAMPP. So, creating my local server with the integrated MariaDB on LattePanda Mu was straightforward and effortless.

#️⃣ First, I downloaded the XAMPP Linux installer (LAMPP).

#️⃣ After downloading the XAMPP installer, I changed its permissions via the terminal (command line).

sudo chmod 755 /home/lattepanda-mu/Downloads/xampp-linux-x64-8.2.12-0-installer.run

#️⃣ Then, I executed the XAMPP installer via the terminal.

sudo /home/lattepanda-mu/Downloads/xampp-linux-x64-8.2.12-0-installer.run

#️⃣ After completing the installation steps, I initiated the XAMPP application (LAMPP) via the terminal to activate the required servers — Apache Web Server and MySQL Database (MariaDB).

sudo /opt/lampp/manager-linux-x64.run

#️⃣ After executing the XAMPP application (LAMPP) on LattePanda Mu successfully, I created the restaurant web application root folder — ai_driven_lora_kiosk — and started to develop the features I envisioned.

#️⃣ To avoid file permission errors while running some packages, I changed the web application root folder permissions constantly via the terminal.

sudo chmod -R 777 /opt/lampp/htdocs

Step 7.1: Setting up PHP Composer for installing third-party packages

PHP Composer is a dependency manager to declare and install third-party code packages, leveraging the built-in PHP class autoloading features, repositories of PHP packages such as Packagist, and common project layout and coding conventions.

#️⃣ First, I needed to establish the cURL package via the terminal to install Composer.

sudo apt-get install curl

#️⃣ Since the XAMPP application (LAMPP) has a built-in PHP-CLI, I was able to download the Composer dependency manager directly into LAMPP via the terminal.

sudo curl -s https://getcomposer.org/installer | /opt/lampp/bin/php

#️⃣ To make Composer system-wide executable, I symlinked (symbolic link) the XAMPP PHP-CLI and moved the composer.phar file to its global destination via the terminal.

sudo ln -s /opt/lampp/bin/php /usr/local/bin/php

sudo mv composer.phar /usr/local/bin/composer

#️⃣ Then, I checked whether Composer runs globally via the terminal.

composer

Step 7.2: Installing the PHP-MQTT client via Composer

Although there are various packages allowing developers to create a PHP-based MQTT client, I decided to utilize this PHP-MQTT client package to build my local MQTT client, operated by the restaurant web application, to connect to The Things Stack MQTT server (broker).

#️⃣ First, I navigated to the folder in which I wanted to download the PHP-MQTT client package via the terminal.

cd /opt/lampp/htdocs/ai_driven_lora_kiosk/assets

#️⃣ Then, I installed the PHP-MQTT client package via Composer.

composer require php-mqtt/client

Step 8: Setting up Ollama on LattePanda Mu

Ollama is a lightweight, extensible, and open-source framework for building and running large language models (LLMs) locally. It provides a simple API for creating, running, and managing LLMs, as well as a vast library of pre-built large language models that can easily be integrated into commercial or personal applications via community-built packages and modules.

In this regard, Ollama is the perfect framework to enable my drive-through restaurant web application to run large language models (LLMs) locally to generate user-specific menus/deals.

#️⃣ I installed the Ollama bundle for Linux via the terminal.

curl -fsSL https://ollama.com/install.sh | sh

Step 8.1: Installing and trying distinguished open-source LLMs via Ollama

After setting up the Ollama bundle successfully, I started to install distinguished large language models (LLMs) from Ollama's vast model library. As explained earlier, I decided to run large language models on LattePanda Mu locally without connecting a GPU — CPU-only. Thus, I needed to select LLMs considering not only their parameter counts, size, and performance but also their compatibility with my system specifications.

During my initial large language model experiments, I did not optimize or fine-tune inputs (objectives), but tried to review their response time and quality via the terminal, operated by my CPU-only machine. After empirically testing LLMs provided by Ollama, I decided to enable these models on the restaurant web application due to their efficiency.

• deepseek-r1:8b [DeepSeek-R1-0528-Qwen3-8B]
• deepseek-r1:7b [DeepSeek-R1-Distill-Qwen-7B]
• deepseek-r1:1.5b [DeepSeek-R1-Distill-Qwen-1.5B]
• gemma3:4b [4B parameter model (128k context window)]
• gemma3:1b [1B parameter model (32k context window)]
• llama3.2:3b [3B parameters]
• qwen3:4b [4B parameter model]
• phi4-mini [128K token context]

#️⃣ Briefly, my initial testing process for each large language model was running them via the terminal and then asking them to define a hamburger to inspect their response time and accuracy.

#️⃣ If the executed LLM has not been installed yet, Ollama downloads the requested model automatically and lets the user discard the chat by entering /bye on the terminal.

ollama run deepseek-r1:7b

ollama run deepseek-r1:1.5b

ollama run gemma3:4b

ollama run llama3.2:3b

ollama run qwen3:4b

ollama run deepseek-r1:8b

ollama run gemma3:1b

ollama run phi4-mini

Step 8.2: Installing the Ollama-PHP helper library via Composer

Since Ollama exposes its web API locally once installed, there are countless methods to access Ollama to run large language models (LLMs). In this regard, I was going to program my own wrapper library in PHP to make Ollama API calls easier to maintain. Nevertheless, I came across a wonderful open-source Ollama-PHP helper library providing lots of features and decided to employ this helper library instead while developing my restaurant web application.

#️⃣ First, I navigated to the folder in which I wanted to download the Ollama-PHP helper library via the terminal.

#️⃣ Although it is possible to install multiple packages into the same vendor folder by modifying the composer.json file, I decided to download the Ollama-PHP library into a separate folder due to my workflow.

cd /opt/lampp/htdocs/ai_driven_lora_kiosk/assets/lib/llm

#️⃣ Then, I installed the Ollama-PHP helper library via Composer.

composer require ardagnsrn/ollama-php

Step 9: Defining the available food items and the generic menus/deals offered by the drive-through restaurant

Before proceeding with developing the restaurant web application, I needed to decide the food items available at my drive-through restaurant establishment so as to use them as the foundation to determine the generic menus/deals offered by the drive-through restaurant and to define the objectives (inputs) for large language models (LLMs) while generating user-specific menus/deals.

#️⃣ First, I created a JSON object containing the available food items (with prices) classified by six food categories. According to my restaurant workstation layout, each food category, hence its items, is assigned to an individual food prep station.

• Burgers ➡ Food prep station [1]
• Mexican ➡ Food prep station [2]
• Italian ➡ Food prep station [3]
• Seafood ➡ Food prep station [4]
• Side Dishes ➡ Food prep station [5]
• Desserts ➡ Food prep station [6]

{
	{
		"category": "Burgers",
	    "items": [
			{"name": "Hamburger", "price": "12"},
			{"name": "Cheeseburger", "price": "10"},
			{"name": "Veggie", "price": "9"}
		]
	},
	{
		"category": "Mexican",
	    "items": [
			{"name": "Taco", "price": "12"},
			{"name": "Burrito", "price": "12"},
			{"name": "Tamales", "price": "12"}
		]
	},
	{
		"category": "Italian",
	    "items": [
			{"name": "Lasagna", "price": "12"},
			{"name": "Risotto", "price": "15"},
			{"name": "Pizza", "price": "20"}
		]
	},
	{
		"category": "Seafood",
	    "items": [
			{"name": "Shrimp Tempura", "price": "8"},
			{"name": "Salmon", "price": "22"},
			{"name": "Lobster", "price": "15"}
		]
	},
	{
		"category": "Side Dishes",
	    "items": [
			{"name": "French Fries", "price": "2"},
			{"name": "Onion Rings", "price": "5"},
			{"name": "Chicken Nuggets", "price": "6"}
		]
	},
	{
		"category": "Desserts",
	    "items": [
			{"name": "Brownie", "price": "3"},
			{"name": "Cheesecake", "price": "4"},
			{"name": "Carrot Cake", "price": "4"}
		]
	}
}

#️⃣ Then, by using the defined food item information as the foundation, I determined generic menus/deals purchasable via the drive-through kiosk interface with or without creating a user account on the restaurant web application.

#️⃣ To access generic menu information easily from the MariaDB database, I utilized the JSON format.

#️⃣ I categorized each generic menu by unique order (menu) tags from f001 to f012 and assigned menu information fitting the drive-through restaurant's cuisine based on the available food items.

• Order tag
• Menu definition
• Menu food item list
• Menu total price
• Menu discount percentage
• Menu discounted price

#️⃣ In addition to the mentioned variables, each menu includes a food prep station road map according to the requested food item number by food category from each food prep station.

• Prep station road map

2%2%0%0%6%2

[items_from_station_1, items_from_station_2, items_from_station_3, items_from_station_4, items_from_station_5, items_from_station_6]

{
	
	{
		"order_tag": "f001",
		"definition": "Friday to Friday: Double Hamburger with French Fries",
		"item_list": [
			{"name": "Hamburger", "category": "Burgers", "amount": 2},
			{"name": "French Fries", "category": "Side Dishes", "amount": 4}
		],
		"total_price": 32,
		"discount_percentage": 5,
		"discounted_price": 30.4,
		"prep_station_road_map": "2%0%0%0%4%0"
	},

	{
		"order_tag": "f002",
		"definition": "Friday to Friday: Double Cheeseburger with Onion Rings",
		"item_list": [
			{"name": "Cheeseburger", "category": "Burgers", "amount": 2},
			{"name": "Onion Rings", "category": "Side Dishes", "amount": 6}
		],
		"total_price": 50,
		"discount_percentage": 15,
		"discounted_price": 42.5,
		"prep_station_road_map": "2%0%0%0%6%0"
	},

	{
		"order_tag": "f003",
		"definition": "Thursday to Thursday: Vegetarian Burger Special",
		"item_list": [
			{"name": "Veggie", "category": "Burgers", "amount": 2},
			{"name": "French Fries", "category": "Side Dishes", "amount": 4}
		],
		"total_price": 26,
		"discount_percentage": 8,
		"discounted_price": 23.92,
		"prep_station_road_map": "2%0%0%0%4%0"
	}, 

	{
		"order_tag": "f004",
		"definition": "Thursday to Thursday: Taco Special with Brownie",
		"item_list": [
			{"name": "Taco", "category": "Mexican", "amount": 2},
			{"name": "Brownie", "category": "Desserts", "amount": 2}
		],
		"total_price": 30,
		"discount_percentage": 6,
		"discounted_price": 28.2,
		"prep_station_road_map": "0%2%0%0%0%2"
	}, 
	
	{
		"order_tag": "f005",
		"definition": "Thursday to Thursday: Burrito Special with Cheesecake",
		"item_list": [
			{"name": "Burrito", "category": "Mexican", "amount": 2},
			{"name": "Cheesecake", "category": "Desserts", "amount": 2}
		],
		"total_price": 32,
		"discount_percentage": 8,
		"discounted_price": 29.44,
		"prep_station_road_map": "0%2%0%0%0%2"
	}, 

	{
		"order_tag": "f006",
		"definition": "Thursday to Thursday: Tamales Special with Carrot Cake",
		"item_list": [
			{"name": "Tamales", "category": "Mexican", "amount": 3},
			{"name": "Carrot Cake", "category": "Desserts", "amount": 3}
		],
		"total_price": 48,
		"discount_percentage": 10,
		"discounted_price": 43.2,
		"prep_station_road_map": "0%3%0%0%0%3"
	}, 

	{
		"order_tag": "f007",
		"definition": "Wednesday to Wednesday: Lasagna Triple Special",
		"item_list": [
			{"name": "Lasagna", "category": "Italian", "amount": 3}
		],
		"total_price": 36,
		"discount_percentage": 5,
		"discounted_price": 34.2,
		"prep_station_road_map": "0%0%3%0%0%0"
	},

	{
		"order_tag": "f008",
		"definition": "Wednesday to Wednesday: Risotto Triple Special",
		"item_list": [
			{"name": "Risotto", "category": "Italian", "amount": 3}
		],
		"total_price": 45,
		"discount_percentage": 10,
		"discounted_price": 40.5,
		"prep_station_road_map": "0%0%3%0%0%0"
	},
	
	{
		"order_tag": "f009",
		"definition": "Wednesday to Wednesday: Pizza Triple Special",
		"item_list": [
			{"name": "Pizza", "category": "Italian", "amount": 3}
		],
		"total_price": 60,
		"discount_percentage": 12,
		"discounted_price": 52.8,
		"prep_station_road_map": "0%0%3%0%0%0"
	},

	{
		"order_tag": "f010",
		"definition": "Sea Weekend Special: Shrimp Tempura",
		"item_list": [
			{"name": "Shrimp Tempura", "category": "Seafood", "amount": 10}
		],
		"total_price": 80,
		"discount_percentage": 15,
		"discounted_price": 68,
		"prep_station_road_map": "0%0%0%10%0%0"
	},

	{
		"order_tag": "f011",
		"definition": "Sea Weekend Special: Salmon and Chips",
		"item_list": [
			{"name": "Salmon", "category": "Seafood", "amount": 5},
			{"name": "French Fries", "category": "Side Dishes", "amount": 10}
		],
		"total_price": 130,
		"discount_percentage": 20,
		"discounted_price": 104,
		"prep_station_road_map": "0%0%0%5%10%0"
	},

	{
		"order_tag": "f012",
		"definition": "Sea Weekend Special: Lobster",
		"item_list": [
			{"name": "Lobster", "category": "Seafood", "amount": 10}
		],
		"total_price": 150,
		"discount_percentage": 30,
		"discounted_price": 105,
		"prep_station_road_map": "0%0%0%10%0%0"
	}
	
}

Step 10: Developing a feature-rich Ollama and TTN-enabled (via MQTT) restaurant web application as the main drive-through kiosk and customer interface on LattePanda Mu

As discussed earlier, I wanted to develop the restaurant web application as the main drive-through kiosk and customer interface, managing customer interactions, processing LoRa-transmitted commands, and running large language models (LLMs) for generating user-specific menus/deals concurrently. In this regard, I needed to develop the web application while working on the automatic food delivery system and the kiosk customer endpoint, including mechanical part design, PCB layout design, etc. Since the web application development process was not linear, I decided to showcase and explain the final version directly. Thus, please refer to the following steps to review all interconnected restaurant features, or you can visit the project GitHub page to inspect the restaurant web application.

Since I wanted to utilize authentic visual assets for the restaurant web application's menu pictures and graphics, I designed them with the assistance of Adobe Illustrator AI Vector Generator.

The directory structure (alphabetically) of the restaurant web application is as follows, under ai_driven_lora_kiosk as the application root folder:

• /assets
• /img
• /lib
• /llm
• /vendor
• composer.json
• composer.lock
• /vendor
• composer.json
• composer.lock
• /prep_stations
• /apriltag_samples
• /jpg_converted
• RGB565_raw_buffer_converter.py
• food_items_by_prep_table.json
• generic_menu_information_by_item.json
• /script
• dashboard_config.js
• dashboard_status_update.js
• order_schedule_update.js
• /style
• dashboard_style.css
• dashboard_style_root_settings.css
• order_schedule.css
• account_config.php
• class.php
• create_necessary_database_tables.sql
• dashboard_status_update.php
• database_secrets.php
• food_delivery_system_conn.php
• order_schedule_update.php
• generate_LLM_based_user_specific_deals.php
• index.php
• order_schedule.php
• ttn_mqtt_client.php

Step 10.1: Constructing the necessary database tables on MariaDB

Since the XAMPP application (LAMPP) integrates the phpMyAdmin software for handling server-based MySQL administration, it is relatively easy to create databases and construct database tables.

#️⃣ First, I wrote all the SQL queries I needed to construct the required database tables and stored them in the create_necessary_database_tables.sql file.

#️⃣ Then, I opened phpMyAdmin on the browser and created the ai_lora_kiosk_user_data database.

localhost/phpmyadmin/

#️⃣ I constructed all the required database tables and inserted the necessary food item and generic menu information by executing the stored SQL queries.

#️⃣ As mentioned earlier, I created the available food item and generic menu information in the JSON format. Thus, I was able to register JSON objects directly to the associated MariaDB database tables column by column.

Step 10.2: Thorough file-by-file code documentation of the restaurant web application

📁 database_secrets.php

Define the MariaDB database server connection settings via the MySQLi extension.

$server = array(
	"server" => "localhost",
	"username" => "root",
	"password" => "",
	"database_name" => "ai_lora_kiosk_user_data"
);

// Database connection credentials.
$_db_conn = mysqli_connect($server["server"], $server["username"], $server["password"], $server["database_name"]);

📁 class.php

To create a concise backend code structure, I compartmentalize functions by extending the primary class — db_kiosk. The subclasses inherit all of the public and protected methods from their parent, even if the parent is a subclass itself.

📌 db_kiosk

⭐ Define the required MariaDB database table information and the default selected LLM for user accounts.

protected $db_conn;
	protected $lora_task_table = "lora_user_task_log", $user_info_table = "user_info", $menu_info_table = "menu_info_by_user", $current_user_table = "current_kiosk_user", $food_items_table = "food_items_by_prep_station";
	
	private $default_LLM = "llama3.2:3b";
	
	public function __init__($_db_conn){
		// Init the MySQL object with the passed database settings.
		$this->db_conn = $_db_conn;
	}

⭐ In the obtain_user_information function, get the registered customer information using the unique account authentication key.

public function obtain_user_information($auth_key){
		$sql = "SELECT * FROM `$this->user_info_table` WHERE `authentication_key` = '$auth_key'";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			// If found successfully, return the registered user information.
			if($row = mysqli_fetch_assoc($result)){
				return $row;
			}else{
				return false;
			} 
		}else{
			return false;
		}		
	}

⭐ In the current_kiosk_user_config function, based on the passed task, change all current user information, update only the account authentication status, or return the stored current user information for further usage.

public function current_kiosk_user_config($db_task, $user_config){
		// Get the current date & time (server).
		$date = date("Y_m_d_h_i_s");
		// Change the current user (customer) information in the given MariaDB database table.
		if($db_task == "change"){
			$sql = "UPDATE `$this->current_user_table`
					SET `authentication_key` = '".$user_config["auth_key"]."',
						`last_change` = '$date',
						`account_status` = '".$user_config["account_status"]."',
						`kiosk_auth_status` = '".$user_config["kiosk_auth_status"]."'
					WHERE `id` = 1;";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;
		}
		// Update the kiosk authentication status (vehicle recognition) status.
		else if($db_task == "update_status"){
			$sql = "UPDATE `$this->current_user_table`
					SET `last_change` = '$date',
						`kiosk_auth_status` = '".$user_config["kiosk_auth_status"]."'
					WHERE `id` = 1;";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;			
		}
		// Return the current user information.
		else if($db_task == "return"){
			$sql = "SELECT * FROM `$this->current_user_table` WHERE `id` = 1;";
			$result = mysqli_query($this->db_conn, $sql);
			$check = mysqli_num_rows($result);
			if($check > 0){
				// If found successfully, return the latest task information.
				if($row = mysqli_fetch_assoc($result)){
					return $row;
				}else{
					return false;
				} 
			}else{
				return false;
			}		
		}		
	}

⭐ In the login_to_user_account function, attempt to obtain the account information for the provided username and password. If found successfully, change the current user (customer) information for further tasks.

public function login_to_user_account($login_info){
		$sql = "SELECT * FROM `$this->user_info_table` WHERE `username` = '".strip_tags(mysqli_real_escape_string($this->db_conn, $login_info['username']))."' and `password` = '".strip_tags(mysqli_real_escape_string($this->db_conn, $login_info['_password']))."'";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			// If found successfully, return the required user information.
			if($row = mysqli_fetch_assoc($result)){
				// After successfully finding the passed user account, change the current user (customer) information for further tasks.
				if($this->current_kiosk_user_config("change", array("auth_key" => $row["authentication_key"], "account_status" => "signed", "kiosk_auth_status" => "pending"))){
					// Return the required info.
					return array("res" => "Given account found successfully!", "auth_key" => $row["authentication_key"]);
				}else{
					return array("res" => "Database error! Status change!", "auth_key" => "");
				}				
			}else{
				return array("res" => "Database error! Information retrieval!", "auth_key" => "");
			} 
		}else{
			return array("res" => "Database error! No user found with the passed credentials!", "auth_key" => "");
		}		
	}

⭐ In the create_new_user_account function:

⭐ Generate the unique 4-digit account authentication key.

⭐ Insert the provided user information and the generated authentication key into the associated MariaDB database table.

⭐ Insert the placeholder user-specific menu/deal information into the given MariaDB database table, classified via the generated authentication key.

public function create_new_user_account($user_info){
		// Generate the authentic account authentication key.
		$auth_key = $this->generate_auth_key(4);
		// Insert the provided user information into the given MariaDB database table.
		$sql = "INSERT INTO `$this->user_info_table` (`firstname`, `lastname`, `email`, `username`,  `password`,  `menu_preference`, `card_holder_name`, `card_number`, `card_exp_date`, `card_cvv`, `activated_LLM`, `authentication_key`)
		        VALUES ('".strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['firstname']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['lastname']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['email']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['username']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['_password']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['menu_preference']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['card_holder_name']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['card_number']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['card_exp_date']))."', '"
		                  .strip_tags(mysqli_real_escape_string($this->db_conn, $user_info['card_cvv']))."', '"
		                  .$this->default_LLM."', '"
						  .$auth_key."');";
		// Exit and notify if the server throws a database error.
		if(!mysqli_query($this->db_conn, $sql)){ return array("res" => "Database error!", "auth_key" => ""); }
				 
        // Insert the placeholder user-specific menu and deal information into the given MariaDB database table.
		$menu_information = array("deal_1" => "pending", "deal_2" => "pending", "deal_3" => "pending", "deal_4" => "pending", "deal_5" => "pending", "deal_6" => "pending", "deal_7" => "pending", "deal_8" => "pending", "deal_9" => "pending", "deal_10" => "pending", "deal_11" => "pending", "deal_12" => "pending");
		$sql_menu_insert = "INSERT INTO `$this->menu_info_table` (`authentication_key`, `deal_1`, `deal_2`, `deal_3`, `deal_4`, `deal_5`, `deal_6`, `deal_7`, `deal_8`, `deal_9`, `deal_10`, `deal_11`, `deal_12`)
							VALUES ('".$auth_key."', '"
									  .$menu_information["deal_1"]."', '"
									  .$menu_information["deal_2"]."', '"
									  .$menu_information["deal_3"]."', '"
									  .$menu_information["deal_4"]."', '"
									  .$menu_information["deal_5"]."', '"
									  .$menu_information["deal_6"]."', '"
									  .$menu_information["deal_7"]."', '"
									  .$menu_information["deal_8"]."', '"
									  .$menu_information["deal_9"]."', '"
									  .$menu_information["deal_10"]."', '"
									  .$menu_information["deal_11"]."', '"
									  .$menu_information["deal_12"]."');";
		// Exit and notify if the server throws a database error.
		if(!mysqli_query($this->db_conn, $sql_menu_insert)){
			return array("res" => "Database error!", "auth_key" => "");
		}else{
			// After successfully creating a new user account with all necessities, change the current user (customer) information for further tasks.
			if($this->current_kiosk_user_config("change", array("auth_key" => $auth_key, "account_status" => "signed", "kiosk_auth_status" => "pending"))){
				return array("res" => "Thanks 😊\nYour account is successfully created, ".$user_info['firstname']."!", "auth_key" => $auth_key);
			}else{
				return array("res" => "Application Error!", "auth_key" => "");
			}			
		}
	}

As mentioned earlier, I decided to transfer the account authentication key from the kiosk customer endpoint to the restaurant web application via LoRaWAN once the customer endpoint identifies the registered vehicle via object detection.

Since LoRaWAN employs hexadecimal encoding while transferring data packets, also referred to as base-16, I had to utilize only the 16 digits of hexadecimal numbers to generate my LoRa-transmitted commands, including but not limited to account authentication keys.

• 0 to 9
• A, B, C, D, E, F

⭐ In the generate_auth_key function, create a unique account authentication key from the suitable characters — hexadecimal digits.

⭐ Assign the A digit as the first character of the account authentication key.

⭐ Then, select randomly from the remaining hexadecimal digits to generate the authentication key.

private function generate_auth_key($len){
		// All authentication keys should start with the 'a' character.
		$auth_key = "a";
		// Define the applicable characters.
		$auth_chars = array("0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "b", "c", "d", "e", "f");
		// Get and assign a random character from the predefined character array.
		for($i=0; $i<$len-1; $i++){
			$auth_key .= $auth_chars[rand(0, count($auth_chars)-1)];
		}
		// Return the generated authentication key.
		return $auth_key;
	}

📌 db_kiosk_llm extends db_kiosk

As mentioned earlier, I created the generic menu/deal information as individual JSON objects. Thus, I needed to make the available large language models (LLMs) output the generated user-specific menus/deals in the same format. Please refer to the following steps to review how I fine-tuned and optimized LLM objectives (inputs) to yield reliable and accurate results.

After rigorously experimenting with the available LLMs, I realized each model utilizes a slightly different JSON syntax while generating the primary JSON object, consisting of user-specific menus/deals. Thus, I modified the produced LLM text responses by considering every JSON syntax I had recorded during my experiments. Even though I had rarely encountered unforeseen syntax aberrations due to the nature of LLMs, I was able to revise the LLM-generated text responses to fetch user-specific menus/deals as individual JSON objects, categorized by unique order (menu) tags from e001 to e012, similar to generic menus/deals.

⭐ In the process_LLM_generated_user_specific_deals function:

⭐ Modify the passed LLM-generated text response to produce the primary JSON object, consisting of user-specific menus/deals.

⭐ Retrieve the available food item categories denoting food prep stations.

⭐ Replace repeating characters (if any) in the modified LLM response, which would cause errors while decoding JSON objects.

⭐ Decode the produced primary JSON object to obtain user-specific menus/deals separately as a multi-dimensional array.

⭐ For each user-specific menu, produce the food prep station road map depending on the requested food item amount per prep station, similar to generic menus/deals. Then, append the produced prep station road map into the associated array element.

⭐ Obtain the current dashboard/kiosk user information to check the customer account authorization status.

⭐ If authorized, encode each user-specific menu from an array element to a JSON object and save the converted JSON objects individually to the associated MariaDB database table row, column by column, classified via the account authentication key that of the current dashboard/kiosk user.

public function process_LLM_generated_user_specific_deals($llm_response){
		// Modify the LLM-generated text to obtain JSON objects (menus) in the required format.
		if(str_contains($llm_response, '```json [') || str_contains($llm_response, '```json[')){
			$llm_response = explode('```json', $llm_response)[1];
			$llm_response = explode('```', $llm_response)[0];
		}else if(str_contains($llm_response, '```json')){
			if(substr_count($llm_response, '```json') == 1){
				$llm_response = str_replace('```json', '```json [', $llm_response);
				$llm_response = str_replace(array('} {', '}{'), '}, {', $llm_response);
				$llm_response = str_replace(array('}```', '} ```'), '} ] ```', $llm_response);
			}else{
				$llm_response_mod = "```json [ ";
				foreach(explode('```json', $llm_response) as $object){
					if(str_contains($object, 'order_tag')){
						$object = explode('```', $object)[0];
						$llm_response_mod .= $object.", ";
					}
				}
				$llm_response_mod .= "] ```";
				$llm_response_mod = str_replace(', ] ```', '] ```', $llm_response_mod);
				$llm_response = $llm_response_mod;
			}
			$llm_response = explode('```json', $llm_response)[1];
			$llm_response = explode('```', $llm_response)[0];
		}else if(str_contains($llm_response, '**')){
			$llm_response = str_replace('**Menu e001**', '```json [', $llm_response);
			for($i=1;$i<13;$i++){
				$m_code = "";
				if($i < 10){ $m_code = "e00".$i; }
				else{ $m_code = "e0".$i; }
				$llm_response = str_replace('**Menu '.$m_code.'**', ',', $llm_response);
			}
			$llm_response .= ' ]';
			$llm_response = explode('```json', $llm_response)[1];
			$llm_response = explode('```', $llm_response)[0];			
		}
		
		// Obtain the available food (menu) item categories by prep station.
		$menu_categories = $this->obtain_available_food_item_info("category");
		
		// Replace repeating characters (if any) causing errors while decoding JSON objects. 
		$llm_response = str_replace(array("::", ": :"), ":", $llm_response);
		
		// Decode the retrieved JSON objects to process menu / deal information accurately.
		$llm_menu_data = json_decode($llm_response, true); // As array.
		// First, produce the prep station road map depending on the requested food item amount per prep station. For instance, 1%0%0%1%2%5 [items from (station_1, station_2, station_2, station_4, station_5, station_6)].
		for($menu_n=0;$menu_n<12;$menu_n++){
			$station_1 = 0; $station_2 = 0; $station_3 = 0; $station_4 = 0; $station_5 = 0; $station_6 = 0;
			foreach($llm_menu_data[$menu_n]["item_list"] as $food_item){
				switch($food_item["category"]){
					case $menu_categories[0]:
						$station_1 += $food_item["amount"];
					break;
					case $menu_categories[1]:
						$station_2 += $food_item["amount"];
					break;	
					case $menu_categories[2]:
						$station_3 += $food_item["amount"];
					break;
					case $menu_categories[3]:
						$station_4 += $food_item["amount"];
					break;
					case $menu_categories[4]:
						$station_5 += $food_item["amount"];
					break;
					case $menu_categories[5]:
						$station_6 += $food_item["amount"];
					break;
					default:
                        // Do nothing.
                    break;						
				
				}
			}
			// Append the produced prep station road map into the passed menu array.
			$prep_station_road_map = $station_1.'%'.$station_2.'%'.$station_3.'%'.$station_4.'%'.$station_5.'%'.$station_6;
			$llm_menu_data[$menu_n]["prep_station_road_map"] = $prep_station_road_map;
		}
		
		// Obtain the current dashboard/kiosk user (customer) information.
		$current_user_info = $this->current_kiosk_user_config("return", array());
		// If the current account has been authorized successfully:
		if($current_user_info["kiosk_auth_status"] == "authorized"){
			// After successfully processing the LLM-generated menu / deal information,
			// convert each menu array to a JSON object in order to update the user-specific menu /deal information that of the current dashboard/kiosk user.
			$column = 0;
			foreach($llm_menu_data as $user_specific_deal){
				// Next column.
				$column++;
				// After encoding the menu array, replace single quotation mark with its HTML entity to avoid errors.
				$user_specific_deal_json = json_encode($user_specific_deal);
				$user_specific_deal_json = str_replace("'", "'",$user_specific_deal_json);
				// Update the user-specific menu /deal information.
				$sql = "UPDATE `$this->menu_info_table`
						SET `deal_".$column."` = '$user_specific_deal_json'
						WHERE `authentication_key` = '".$current_user_info["authentication_key"]."';";
				// Show the query result.
				if(mysqli_query($this->db_conn, $sql)){
					echo "<p>".$user_specific_deal["order_tag"].": registered successfully!</p><br>";
				}else{
					echo "<p>".$user_specific_deal["order_tag"].": database registration error!</p><br>";
				}		
			}			
		}else{
			echo "<p>Account not authorized!</p>";
		}		
	}

⭐ In the obtain_available_food_item_info function:

⭐ Obtain the available food item information distinguished by food prep stations from the associated MariaDB database table.

⭐ Remove the id column to avoid errors.

⭐ If requested, return the retrieved food item information directly (JSON objects) as a string.

⭐ Otherwise, decode the retrieved JSON objects to return only the requested data element for each prep station as an array.

public function obtain_available_food_item_info($type){
		$sql = "SELECT * FROM `$this->food_items_table`";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			if($row = mysqli_fetch_assoc($result)){
				// Remove the id column to avoid errors.
				unset($row["id"]);
				// If requested, return the retrieved food item information without modifying (as JSON objects) to feed the selected LLM as data points.
				if($type == "json"){
					$food_item_info_json = "";
					foreach($row as $prep_table){
						$food_item_info_json .= $prep_table.",";
					}
					return $food_item_info_json;					
				}
				// Otherwise, decode the retrieved JSON objects for each prep station and return only the requested data type in the output array.
				else{
					$output_food_item_info = [];
					foreach($row as $prep_table){
						$prep_table_arr = json_decode($prep_table, true); // As array.
						array_push($output_food_item_info, $prep_table_arr[$type]);
					}
					return $output_food_item_info;					
				}
			}else{
				return false;
			} 
		}else{
			return false;
		}		
	}

⭐ In the obtain_menu_deal_information function:

⭐ According to the passed account authentication key, obtain generic or user-specific menu/deal information from the associated MariaDB database table.

#️⃣ To store generic menus/deals with user-specific menus/deals, I simply assigned generic as the authentication key for the generic menu row.

⭐ Decode the retrieved JSON objects for each menu/deal to create a multi-dimensional array of menu information.

⭐ If requested, return the created menu information array directly.

⭐ Otherwise, modify the generated array to return a subarray of only the requested data element for each menu, classified via the unique order (menu) tags.

public function obtain_menu_deal_information($auth_key, $type){
		$sql = "SELECT * FROM `$this->menu_info_table` WHERE `authentication_key` = '$auth_key'";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			if($row = mysqli_fetch_assoc($result)){
				// Remove the id and authentication_key columns to avoid errors.
				unset($row["id"], $row["authentication_key"]);
				// Decode the retrieved JSON objects for each menu / deal.
				$deals_arr = [];
				foreach($row as $deal){
					$deal_arr = json_decode($deal, true); // As array.
					array_push($deals_arr, $deal_arr);
				}
				// Depending on the passed data type, return the whole deal information or only the necessary information by the order tag.
				if($type == "all"){
					return $deals_arr;
				}else{
					$necessary_info_arr = [];
					for($i=0;$i<count($deals_arr);$i++){
						$necessary_info_arr[$deals_arr[$i]["order_tag"]] = $deals_arr[$i][$type];
					}
					return $necessary_info_arr;
				}
			}else{
				return false;
			}
		}else{
			return false;
		}
		
	}

📌 db_kiosk_lora extends db_kiosk_llm

⭐ In the obtain_latest_lora_log function:

⭐ Fetch the latest LoRa-transmitted task log entry from the associated MariaDB database table according to the passed task type.

⭐ Then, generate an array of the retrieved LoRa-transmitted task log and return it.

public function obtain_latest_lora_log($task_type){
		$lora_log_info = false;
		// Obtain the latest LoRa task log information according to the requested task type.
		$query = ($task_type != "all") ? "WHERE `task_type` = '$task_type'" : "";
		$sql = "SELECT * FROM `$this->lora_task_table` ".$query." ORDER BY `log_id` DESC";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			// Generate the required LoRa task log array.
			if($row = mysqli_fetch_assoc($result)){
				$lora_log_info = array( 
				                        "device_id" => $row["device_id"],
				                        "gateway_id" => $row["gateway_id"],
										"lora_code" => $row["decoded_payload"],
										"task_type" => $row["task_type"],
										"task_details" => $row["task_details"],
										"task_status" => $row["task_status"],
										"up_date" => $row["server_time"]
									  );						
			}
		}
		// Return the retrieved LoRa log information.
		return $lora_log_info;		
	}

⭐ In the append_lora_message_log function:

⭐ Insert the passed LoRa-transmitted message information as the latest LoRa task (transmission) log into the associated MariaDB database table, including LoRaWAN gateway, end device, and application information provided by The Things Stack MQTT server.

public function append_lora_message_log($lora_message_log){
		// Insert the recent LoRa message information into the given MariaDB database table.
		$sql = "INSERT INTO `$this->lora_task_table` (`device_id`, `application_id`, `gateway_id`, `received_at`,  `frm_payload`,  `decoded_payload`, `task_type`, `task_details`, `task_status`, `server_time`)
		        VALUES ('".$lora_message_log['device_id']."', '"
				          .$lora_message_log['application_id']."', '"
						  .$lora_message_log['gateway_id']."', '"
				          .$lora_message_log['received_at']."', '"
						  .$lora_message_log['frm_payload']."', '"
						  .$lora_message_log['decoded_payload']."', '"
						  .$lora_message_log['task_type']."', '"
						  .$lora_message_log['task_details']."', '"
						  .$lora_message_log['task_status']."', '"
						  .$lora_message_log['server_time']."');";
		// Show the query result.
	    return (mysqli_query($this->db_conn, $sql)) ? true : false;
	}

⭐ In the decode_lora_message function:

⭐ Obtain the Apache web server's local time.

⭐ Decode the passed JSON object (LoRa-transmitted message information) as a PHP object.

Even though the LoRaWAN protocol utilizes the hexadecimal encoding, The Things Stack sets frm_payload to the Base64 representation of the binary payload (hex encoded message) while transferring data packets via its integrated MQTT server (broker).

#️⃣ Thus, I needed to convert the Base64 encoded frm_payload to a hexadecimal string to obtain the original uplink message (task) transferred by the kiosk customer endpoint.

⭐ Convert the retrieved frm_payload (Base64 encoded) to a hexadecimal string (Base64 -> ASCII -> Hex) so as to extrapolate the transferred uplink message.

According to the first character of the received 4-digit hexadecimal string (LoRa-transmitted data packet), there are three different primary task categories.

• a___ [User Account Authentication Key]
• e___ [User-specific (LLM-generated) Menu/Deal Order Tag]
• f___ [Generic Menu/Deal Order Tag]

🚩 'a'

⭐ When the kiosk customer endpoint transfers an account authentication key via LoRaWAN, compare the received key with the current dashboard/kiosk user's authentication key.

⭐ If they match and the current user account has not already been authorized, update the current dashboard/kiosk user information accordingly to enable the current user account to generate user-specific menus/deals and place orders for them.

🚩 'e'

⭐ Once the customer sends a user-specific menu order tag through the kiosk customer endpoint via LoRaWAN, obtain the current dashboard/kiosk user information to check the account authorization status.

⭐ If so, obtain all of the food prep station road maps of user-specific menus from the associated MariaDB database table, classified by the current user's account authentication key.

⭐ Then, check whether the requested user-specific menu exists and is ready for delivery.

⭐ If so, insert the requested menu information into the associated MariaDB database table to update the food delivery system log accordingly, including the menu's food prep station road map, the menu order tag, and the current user's account authentication key.

🚩 'f'

⭐ As mentioned earlier, generic menus/deals are available to purchase with or without a user account. So, once the customer sends a generic menu order tag through the kiosk customer endpoint via LoRaWAN, obtain all of the food prep station road maps of generic menus from the associated MariaDB database table directly.

⭐ Then, check whether the requested generic menu exists and is ready for delivery.

⭐ If so, insert the requested menu information into the associated MariaDB database table to update the food delivery system log accordingly, including the menu's food prep station road map and the menu order tag.

⭐ After decoding the received LoRa message and concluding the requested task, produce the LoRa uplink message log array, including LoRaWAN gateway, end device, and application information provided by The Things Stack MQTT server. Then, return the generated log array for further usage.

public function decode_lora_message($lora_message){
		// Get the current date & time (server).
		$date = date("Y_m_d_h_i_s");
		// Decode the passed JSON object (LoRa message).
		$lora_message = json_decode($lora_message);
		// Convert the retrieved frm_payload (Base64 encoded) to hexadecimal string (Base64 -> ASCII -> Hex) so as to extrapolate the transferred uplink message value.
		$payload = bin2hex(base64_decode($lora_message->uplink_message->frm_payload));
		/*
			Then, obtain the requested task depending on the first character of the retrieved hexadecimal string — 4-digit.
				a___ [User Account Authentication Code]
				e___ [User-specific Menus and Deals — LLM-generated]
				f___ [Generic Menus and Deals]				
		*/
		$task_type = ""; $task_details = ""; $task_status = "";
		switch($payload[0]){
			case "a":
				$task_type = "auth_account";
				$task_details = "Kiosk sent the account authentication code.";
				// When the user sends the account authentication code, compare it with the registered dashboard/kiosk user information.
				$current_user_info = $this->current_kiosk_user_config("return", array());
				if($current_user_info != false){
					// If the authentication code is correct and the current account has not already been authorized,
					// update the dashboard/kiosk user information accordingly to enable the current user account to place orders.
					if($payload == $current_user_info["authentication_key"] && $current_user_info["kiosk_auth_status"] != "authorized"){	
						$this->current_kiosk_user_config("update_status", array("kiosk_auth_status" => "authorized"));
						$task_status = "Account has been authorized successfully";
					}else{
						$task_status = "Given auth code is wrong or the account has already been authorized!";
					}
				}else{
					$task_status = "Database error [compare current user]!";
				}				  
			break;
			case "e":
				$task_type = "order_specific";
				$task_details = "Customer purchased a user-specific menu / deal generated by the selected LLM based on the passed deal preference.";
				/* When the user sends a menu tag to purchase a user-specific menu / deal (generated by the selected LLM), update the latest task information accordingly. */
				// First, obtain the current dashboard/kiosk user (customer) information.
				$current_user_info = $this->current_kiosk_user_config("return", array());
				if($current_user_info != false){
					// If the current user account is successfully authorized, obtain the prep station road map that of the requested menu.
					if($current_user_info["kiosk_auth_status"] == "authorized"){
						$prep_station_road_maps = $this->obtain_menu_deal_information($current_user_info["authentication_key"], "prep_station_road_map");
						// Check whether the retrieved menu information is correct or not.
						if($prep_station_road_maps != false && array_key_exists($payload, $prep_station_road_maps)){
							// If the menu / deal information is fetched successfully, transfer the assigned prep station road map to the food delivery system.
							$menu_prep_road_map = $prep_station_road_maps[$payload];
							if($this->food_delivery_system_log("insert", array("task_type" => $task_type, "task_objectives" => $menu_prep_road_map, "task_status" => "initiated", "authentication_key" => $current_user_info["authentication_key"], "order_tag" => $payload))){
								$task_status = "Requested user-specific menu / deal [LLM-generated] order information successfully transferred to the food delivery system!"; /* 2%2%0%0%6%2 [station_1, station_2, station_3, station_4, station_5, station_6] */
							}else{
								$task_status = "Cannot transfer information to the food delivery system!";
							}							
						}else{
							$task_status = "There is no menu / deal with the passed menu tag.";
						}
					}else{
						$task_status = "User account has not been authorized yet!";
					}
				}else{
					$task_status = "Cannot access the current user information!";
				}
			break;			
			case "f":
				$task_type = "order_generic";
				$task_details = "Customer purchased a generic menu / deal.";
				/* When the user sends a menu tag to purchase a generic menu / deal, update the latest task information accordingly. */
				// First, obtain the prep station road map that of the requested menu.
				$prep_station_road_maps = $this->obtain_menu_deal_information("generic", "prep_station_road_map");
				// Check whether the retrieved menu information is correct or not.
				if($prep_station_road_maps != false && array_key_exists($payload, $prep_station_road_maps)){
					// If the menu / deal information is fetched successfully, transfer the assigned prep station road map to the food delivery system.
					$menu_prep_road_map = $prep_station_road_maps[$payload];
					if($this->food_delivery_system_log("insert", array("task_type" => $task_type, "task_objectives" => $menu_prep_road_map, "task_status" => "initiated", "authentication_key" => "generic", "order_tag" => $payload))){
						$task_status = "Requested generic menu / deal order information successfully transferred to the food delivery system!"; /* 3%0%0%0%6%0 [station_1, station_2, station_3, station_4, station_5, station_6] */
					}else{
						$task_status = "Cannot transfer information to the food delivery system!";
					}					
				}else{
					$task_status = "There is no menu / deal with the passed menu tag.";
				}
			break;
            default:
				$task_type = "not_recognized"; $task_details = "LoRa message sent by the kiosk is not an assigned task."; $task_status = "idle";
		}

        // Generate the LoRa uplink message log	array with the extrapolated task information.
		$lora_message_log = array(
								   "device_id" => $lora_message->end_device_ids->device_id,
								   "application_id" => $lora_message->end_device_ids->application_ids->application_id,
								   "gateway_id" => $lora_message->uplink_message->rx_metadata[0]->gateway_ids->gateway_id,
								   "received_at" => $lora_message->uplink_message->rx_metadata[0]->received_at,
								   "frm_payload" => $lora_message->uplink_message->frm_payload,
								   "decoded_payload" => $payload,
								   "task_type" => $task_type,
								   "task_details" => $task_details,
								   "task_status" => $task_status,
								   "server_time" => $date
								 );		
		// Return the generated LoRa message log.
		return $lora_message_log;
	}

⭐ In the food_delivery_system_log function:

⭐ Obtain the Apache web server's local time.

⭐ According to the passed operation type for managing the food delivery system log:

⭐ Insert the passed delivery task information based on the requested menu information into the associated MariaDB database table.

⭐ Update the ongoing delivery task information in the MariaDB database table.

⭐ Return the latest delivery task information from the MariaDB database table.

public function food_delivery_system_log($db_command, $task_info){
		// Get the current date & time (server).
		$date = date("Y_m_d_h_i_s");
		// Insert the provided task information into the given MariaDB database table.		
		if($db_command == "insert"){
			$sql = "INSERT INTO `$this->food_delivery_table` (`task_type`, `task_objectives`, `task_status`, `authentication_key`, `order_tag`, `server_time`)
					VALUES ('".$task_info["task_type"]."', '"
							  .$task_info["task_objectives"]."', '"
							  .$task_info["task_status"]."', '"
							  .$task_info["authentication_key"]."', '"
							  .$task_info["order_tag"]."', '"
							  .$date."');";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;	
		}
		// Update the ongoing task information.	
		else if($db_command == "update"){
			$sql = "UPDATE `$this->food_delivery_table`
					SET `task_status` = '".$task_info["task_status"]."',
						`server_time` = '$date'
					ORDER BY `id` DESC LIMIT 1;";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;			
		}
		// Return the ongoing task information.
		else if($db_command == "get"){
			$sql = "SELECT * FROM `$this->food_delivery_table` ORDER BY `id` DESC";
			$result = mysqli_query($this->db_conn, $sql);
			$check = mysqli_num_rows($result);
			if($check > 0){
				// If found successfully, return the latest task information.
				if($row = mysqli_fetch_assoc($result)){
					return $row;
				}else{
					return false;		
				} 
			}else{
				return false;
				
			}			
		}
	}

📌 db_kiosk_status_upt extends db_kiosk_lora

⭐ In the generate_dashboard_updates function:

⭐ Obtain status states and logs required by the web application dashboard by utilizing the preceding functions and return them as a JSON object:

• Account authorization status
• Selected (enabled) LLM
• Latest food delivery system log
• Latest LoRa task (transmission) log

public function generate_dashboard_updates(){
		$kiosk_status = "not_found";
		$enabled_LLM = "not_found";
		$latest_order_log_info = "not_found";
		$latest_lora_log_info = "not_found";
		// Get the current dashboard/kiosk user information.
		$current_user_info = $this->current_kiosk_user_config("return", []);
		// Continue if the database returns the current dashboard/kiosk user information successfully.
		if($current_user_info != false){
			// Kiosk status.
			$kiosk_status = $current_user_info["kiosk_auth_status"];
			// Get the current user (account) information.
			$user_info = $this->obtain_user_information($current_user_info["authentication_key"]);
			// Continue if the database returns the current user (account) information successfully.
			if($user_info != false){
				// Activated LLM.
				$enabled_LLM = $user_info["activated_LLM"];
				// Get the latest LoRa log information from the registered LoRa entries, including error messages.
				$lora_log = $this->obtain_latest_lora_log("all");
				// Continue if the database returns the latest LoRa task log successfully.
				if($lora_log != false){
					// Latest LoRa task log.
					$latest_lora_log_info = $lora_log;
					// Get the latest ordered menu / deal information — generic or user-specific.
					$food_log_info = $this->food_delivery_system_log("get", []);	
					// Continue if the database returns the latest food delivery system log successfully.
					if($food_log_info != false){
						// Latest order log.
						$latest_order_log_info = array("order_type" => $food_log_info["task_type"], "order_tag" => $food_log_info["order_tag"], "order_status" => $food_log_info["task_status"], "order_upt_time" => $food_log_info["server_time"]);
					}					
				}				
			}
		}
		// Return the collected necessary dashboard updates as a JSON object.
		return json_encode(
		                   array("kiosk_status" => $kiosk_status,
						         "enabled_LLM" => $enabled_LLM,
								 "latest_order_log_info" => $latest_order_log_info,
								 "latest_lora_log_info" => $latest_lora_log_info
								 )
						  );
	}

⭐ In the update_selected_LLM function, update the current dashboard/kiosk user's enabled LLM in the associated MariaDB database table.

public function update_selected_LLM($selected_LLM){
		// Get the current dashboard/kiosk user information.
		$current_user_info = $this->current_kiosk_user_config("return", []);
		// Continue if the database returns the current dashboard/kiosk user information successfully.
		if($current_user_info != false){
			// Update the selected LLM of the fetched user.
			$sql = "UPDATE `$this->user_info_table`
					SET `activated_LLM` = '$selected_LLM'
					WHERE `authentication_key` = '".$current_user_info["authentication_key"]."';";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;
		}		
	}

⭐ In the update_deal_preference function, update the current dashboard/kiosk user's deal (menu) preference in the associated MariaDB database table.

public function update_deal_preference($given_preference){
		// Get the current dashboard/kiosk user information.
		$current_user_info = $this->current_kiosk_user_config("return", []);
		// Continue if the database returns the current dashboard/kiosk user information successfully.
		if($current_user_info != false){
			// Update the deal preference of the fetched user.
			$sql = "UPDATE `$this->user_info_table`
					SET `menu_preference` = '$given_preference'
					WHERE `authentication_key` = '".$current_user_info["authentication_key"]."';";
			// Show the query result.
			return (mysqli_query($this->db_conn, $sql)) ? true : false;
		}		
	}

📌 db_kiosk_status_order extends db_kiosk_lora

⭐ In the generate_order_schedule_updates function:

⭐ Obtain all the food delivery system log entries from the associated MariaDB database table.

⭐ To create an HTML table row for each retrieved delivery system log, follow this process:

⭐ First, add the initial columns (parameters).

⭐ Then, obtain all the menu information from the MariaDB database table classified by the passed account authentication key. As mentioned earlier, generic is assigned as the authentication key to each generic menu/deal.

⭐ Retrieve the available food item categories denoting food prep stations.

⭐ Similar to the food prep station road map, generate a food item list [amount x name] for each prep station and add them to the HTML table row as columns.

⭐ Finally, add the remaining parameters (columns) to complete the table row.

⭐ After generating HTML table rows for each delivery system log, return them as a single string.

public function generate_order_schedule_updates(){
		$order_status_html_content = '';
		// First, obtain all of the food delivery system logs.
		$sql = "SELECT * FROM `$this->food_delivery_table` ORDER BY `id` DESC";
		$result = mysqli_query($this->db_conn, $sql);
		$check = mysqli_num_rows($result);
		if($check > 0){
			while($row = mysqli_fetch_assoc($result)){
				$html_content = '';
				// If there are food delivery system logs, fetch the menu / deal list of the given customer account via its authentication key.
				$auth_key = $row["authentication_key"];
				$menu_tag = $row["order_tag"];
				// Define the initial HTML table columns.
				$html_content .= '<tr class="'.$row["task_status"].'"><td>'.$auth_key.'</td><td>'.$menu_tag.'</td><td>'.$row["task_type"].'</td>';
				$menu_information = $this->obtain_menu_deal_information($auth_key, "all");
				// Then, obtain the available menu categories by prep station.
				$menu_categories = $this->obtain_available_food_item_info("category");
				// If the menu / deal list and the available menu categories are retrieved successfully, obtain the requested menu / deal information (generic or user-specific) via its provided menu tag.
				if($menu_information != false && $menu_categories != false){
					// According to the retrieved information, generate the required HTML table content.
					// List items per prep station.
					$station_prep_item_lists = ['', '', '', '', '', ''];
					foreach($menu_information as $menu){
						if($menu["order_tag"] == $menu_tag){
							foreach($menu["item_list"] as $item){
								for($i=0; $i<count($menu_categories); $i++){
									if($item["category"] == $menu_categories[$i]){
										$station_prep_item_lists[$i] .= $item["amount"].' x '.$item["name"].'<br>';
									}
								}
							}
						}
					}
					// Define HTML table columns of prep station items.
					foreach($station_prep_item_lists as $station_item_list){
						if($station_item_list != ''){
							$html_content .= '<td>'.$station_item_list.'</td>';
						}else{
							$html_content .= '<td>No items required!</td>';
						}
					}
					// Define the HTML table columns for the remaining parameters.
					$html_content .= '<td>'.$row["task_status"].'</td><td>'.$row["server_time"].'</td></tr>';
					// Finally, add the produced HTML table row to the HTML table content.
                    $order_status_html_content .= $html_content;			
				}else{
					return false;
				}
			} 
			// After generating all HTML table rows successfully, return the HTML table content.
			return $order_status_html_content;
		}else{
			return false;		
		}
	}

📁 ttn_mqtt_client.php

⭐ Include the class.php file to integrate the required functions.

require "assets/class.php";

⭐ Include the PHP-MQTT client package and add the required modules to program a PHP-MQTT client for only subscribing to the uplink MQTT topics.

⭐ Define the required end device information and MQTT credentials to connect this PHP-MQTT client to The Things Stack MQTT server (broker).

Please refer to the previous steps to review how to configure The Things Stack MQTT server integration settings.

#️⃣ As explained earlier, TTS MQTT server exposes MQTT topics for each application by utilizing the unique application ID and the registered end device ID. Even though TTS MQTT server provides various topics for the uplink traffic, I was able to obtain all the information I needed by subscribing to this topic only:

• v3/{application id}@{tenant id}/devices/{device id}/up

#️⃣ As stated by The Things Stack, TTS MQTT server supports the MQTT Standard Version 3.1.1 and QoS 0 only. Thus, I utilized these protocols to program my PHP-MQTT client accurately.

$ttn_server = array(
				"host" => "au1.cloud.thethings.network",
				"port" => 1883,
				"username" => "kiosk-customer-end1@ttn",
				"password" => "<TTN_PASSWORD>", // e.g., NNSXS.TCN__________________________.JMB_________________________________
				"device_id" => "kiosk-customer-end-device1",
				"version" => MqttClient::MQTT_3_1_1,
				"client_id" => "kiosk_web_app"
			  );
$ttn_topic_up = "v3/".$ttn_server["username"]."/devices/".$ttn_server["device_id"]."/up";

⭐ Enable a new instance of the MqttClient class, which is the backbone of this PHP-MQTT client.

⭐ Create a new ConnectionSettings instance based on the MQTT credentials provided by The Things Stack MQTT server (broker).

⭐ With the defined connection settings, try to establish a connection with TTS MQTT server.

⭐ Disable the clean session since it is beneficial for a QoS level higher than 0.

⭐ Subscribe to the given uplink traffic topic exposed by TTS MQTT server, using QoS 0.

⭐ Include the required MariaDB database secrets and define the db_kiosk_lora class object.

#️⃣ I included the database secrets and defined the class object in this function instead of adding them globally to avoid variable scope errors due to the ongoing MQTT client event loop.

⭐ After receiving the LoRa-transmitted message (uplink) information from TTS MQTT server, decode and process the obtained data packet to perform the requested task and generate the LoRa task (transmission) log.

⭐ Then, save the generated LoRa transmission log to the associated MariaDB database table.

⭐ After configuring the PHP-MQTT client's topic subscription settings, activate the MQTT client event loop to receive the published messages from TTS server (broker) immediately and continuously.

⭐ Terminate the MQTT server connection outside of the event loop.

⭐ Catch and display MQTT server connection errors for debugging.

try{	
	// Enable a new instance of the MqttClient class to establish a connection with the given TTN MQTT server.
	$ttn_client = new MqttClient($ttn_server["host"], $ttn_server["port"], $ttn_server["client_id"], $ttn_server["version"]);
	
	// Configure the connection settings required by the TTN MQTT server depending on the registered TTN application and end device.
	$ttn_connection_setttings = (new ConnectionSettings)
		->setUsername($ttn_server["username"])
		->setPassword($ttn_server["password"]);

	// Attempt to establish a connection with the TTN MQTT server. Disable clean session.
	$ttn_client->connect($ttn_connection_setttings, false);

	// Subscribe to the TTN uplink message topic using QoS 0.
	$ttn_client->subscribe($ttn_topic_up, function(string $topic, string $message, bool $retained){
	    // Include the database settings separately to avoid variable scope errors.
	    include "assets/database_secrets.php";
	        
		// Define the db_kiosk_lora class object in this function to avoid variable errors due to the ongoing MQTT loop.
		$db_kiosk_lora_obj = new db_kiosk_lora(); 
		$db_kiosk_lora_obj->__init__($_db_conn);
		
		// After getting the transferred uplink message from the connected TTN MQTT server, decode the retrieved information to generate the LoRa message log.
		$lora_message_log = $db_kiosk_lora_obj->decode_lora_message($message);
		printf("\n\nTopic => [%s] => Uplink message is received from the connected TTN MQTT server!\n\nLoRa message log is successfully generated!", $topic);
		// Then, save the generated LoRa message log to the given database table with the extrapolated task information.
		if($db_kiosk_lora_obj->append_lora_message_log($lora_message_log)){
			printf("\nLoRa message log is successfully registered to the database!"); 
		}else{
			printf("\nDatabase Error: LoRa message log cannot be saved to the database!");
		}		
	}, MqttClient::QOS_AT_MOST_ONCE);
	
	// Activate the client loop to wait and receive the published messages immediately.
	$ttn_client->loop(true);

	// Terminate the server connection.
	$ttn_client->disconnect();

// Catch and log MQTT server connection errors for debugging.	
}catch(MqttClientException $e){
	printf("MQTT Connection Error => [%s]", $e);
}

📁 generate_LLM_based_user_specific_deals.php

⭐ Enable showing all errors in the browser for debugging.

error_reporting(E_ALL);
ini_set('display_errors', '1');

⭐ Include the Ollama-PHP helper library and add the required modules to manage large language model (LLM) chats.

require "assets/lib/llm/vendor/autoload.php";

// Add the required classes from the ollama-php/client library.
use ArdaGnsrn\Ollama\Ollama;
use ArdaGnsrn\Ollama\Responses\Chat\ChatMessageResponse;
use ArdaGnsrn\Ollama\Responses\Chat\ChatResponse;
use ArdaGnsrn\Ollama\Responses\StreamResponse;

⭐ Include the class.php file to integrate the required functions and define the db_kiosk_llm class object.

require "assets/class.php";

// Define the db_kiosk_llm class object.
$db_kiosk_llm_obj = new db_kiosk_llm(); 
$db_kiosk_llm_obj->__init__($_db_conn);

⭐ Declare the parameters globally, necessary for generating user-specific menus/deals with the current dashboard/kiosk user's selected large language model (LLM).

⭐ Obtain the required information of the current dashboard/kiosk user account.

$customer_selected_LLM = false;
$customer_deal_preference = false;
$available_food_items_and_categories = false;

// Obtain the current customer account information.
$current_user_info = $db_kiosk_llm_obj->current_kiosk_user_config("return", array());
if($current_user_info != false){
	$user_info = $db_kiosk_llm_obj->obtain_user_information($current_user_info["authentication_key"]);
	if($user_info != false){
		$customer_selected_LLM = $user_info["activated_LLM"];
		$customer_deal_preference = $user_info["menu_preference"];		
	}
}

⭐ Obtain the available food item information as JSON objects, distinguished by food categories denoting food prep stations.

⭐ As discussed earlier, I created generic menus/deals as JSON objects. Thus, I defined a blank JSON object with the same structure to make LLMs generate user-specific menus/deals accurately.

$available_food_items_and_categories = $db_kiosk_llm_obj->obtain_available_food_item_info("json");

// Define the required JSON object format for each LLM-generated menu / deal.
$generated_menu_json_format = '{
									"order_tag": "",
									"definition": "",
									"item_list": [
										{"name": "", "category": "", "amount": },

									],
									"total_price": ,
									"discount_percentage": ,
									"discounted_price": 
								} ';

⭐ If all the necessary information is fetched successfully, declare LLM system prompts and the primary task with essential objectives (inputs) to generate user-specific menus/deals based on the provided customer configurations and deal preference.

#️⃣ As mentioned earlier, this is the final version of the web application, and I conducted a lot of experiments with different iterations of LLM system prompts and objectives (inputs) to obtain accurate and reliable results for each available LLM. Thus, please refer to the following steps to review my initial testing of running Ollama via the web application.

⭐ Create a new Ollama-PHP client instance based on the built-in Ollama chat API.

⭐ Then, initiate a new chat session with the selected LLM and feed the declared system prompts and objectives (inputs) to produce user-specific menus/deals in accordance with the provided customer configurations and deal preference.

#️⃣ During my experiments, I noticed that thinking models generate inconsistent results or incompatible JSON objects for my straightforward restaurant menu generation use case since they gaslight themselves to tangential outcomes instead of following the given objectives for utilizing predefined food item information in JSON format. In this regard, I disabled thinking via the built-in Ollama chat API while running the available LLMs.

#️⃣ After a long hour of debugging to find why my code processing the LLM-generated text was not working, I realized that LLMs include escaped (hidden) character sequences that do not appear on the terminal or the browser. Thus, I modified the LLM-generated text to remove these hidden characters.

⭐ Remove the escaped character sequences from the LLM-generated text to avoid errors while parsing the text as JSON objects.

⭐ For further debugging, print the modified LLM-generated response by applying the json_encode function to display the remaining hidden characters, if any.

⭐ Finally, process the modified LLM-generated text response to obtain 12 different user-specific menus/deals as JSON objects and register them to the associated MariaDB database table.

if($customer_selected_LLM != false && $customer_deal_preference != false && $available_food_items_and_categories != false){
	// Define the system requirements for the selected LLM.
	$llm_system_requirements = 'Answer as a drive-through restaurant kiosk computer.';
	// Define the primary task and necessary objectives for the selected LLM based on the provided customer configurations and deal preference.
	$llm_task_and_objectives = '
								Generate 12 different menus using food items from these JSON objects:
								
								'.$available_food_items_and_categories.'
								
								Each menu has to include at least one item from the category '.$customer_deal_preference.'.
								You must add additional items from the category Side Dishes and Desserts to each menu.
								Each menu has to include up to 5 different items.
								You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category '.$customer_deal_preference.'.
								You must calculate the total menu price by using the provided food item prices and the assigned amounts.
								You must assign a discount percentage from 2 percent to 8 percent for each menu.
								You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage.
								You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words.
								You must assign an order tag for each of the 12 different menus, from e001 to e012.
								
								You must generate and return 12 different menus and return them as JSON objects in this format:				
								
								'.$generated_menu_json_format;

    echo '<p>Generated LLM objectives:</p><br><p>'.$llm_task_and_objectives.'</p>';
	

	// Create a new Ollama PHP client based on the built-in Ollama chat API.
	$client = \ArdaGnsrn\Ollama\Ollama::client();

	// Create a new chat session with the passed LLM to generate user-specific menus / deals with the provided customer configurations and deal preference.
    /*
		According to my experiments concerning this use case, thinking models tend to generate inconsistent results or incompatible JSON objects since they gaslight themselves to tangential outcomes for simple or straightforward tasks such as restaurant menu generation.
		Thus, I disabled the thinking feature via the built-in Ollama chat API.
    */

	$response = $client->chat()->create([
		'model' => $customer_selected_LLM,
		'messages' => [
			['role' => 'system', 'content' => $llm_system_requirements],
			['role' => 'user', 'content' => $llm_task_and_objectives]
		],
		'think' => false // Disable thinking to get more accurate results with the given use case.
	]);

	echo "<br><p>Selected (Utilized) LLM: ".$customer_selected_LLM."</p><br>";

	echo "<p>Thinking: Disabled!</p><br>";
	
	echo "<p>Deal Preference: ".$customer_deal_preference."</p><br>";
       
	// Remove the escaped character sequences from the LLM-generated text to avoid errors while parsing the text as JSON objects.
	$remove_escaped_LLM_response = str_replace(array("\n", "\r", "\t"), " ", $response->message->content);
	// Print the response with all hidden characters for debugging.
	echo "<p>LLM Response (text/plain): ".json_encode($remove_escaped_LLM_response)."</p><br>";
	
	// Process the modified LLM-generated response, consisting of all 12 menus / deals as JSON objects, to register each menu / deal to the associated database table.
	$db_kiosk_llm_obj->process_LLM_generated_user_specific_deals($remove_escaped_LLM_response);
    
}

📁 food_delivery_system_conn.php

I programmed this page as the webhook to establish the connection between the restaurant web application and the automatic food delivery system based on Arduino Nicla Vision.

⭐ Include the class.php file to integrate the required functions and define the db_kiosk_lora class object.

require "class.php";

// Define the db_kiosk_lora class object.
$db_kiosk_lora_obj = new db_kiosk_lora(); 
$db_kiosk_lora_obj->__init__($_db_conn);

⭐ Once Nicla Vision requests, return the latest still ongoing food delivery system log information.

!Task (Order) Type&Task Objectives [Prep Station Road Map]&Task Status!

!order_specific&2%0%0%0%6%0&initiated!

if(isset($_GET["get_latest_food_system_log"])){
	$log = $db_kiosk_lora_obj->food_delivery_system_log("get", array());
	if($log != false){
		if($log["task_status"] == "initiated") echo "!".$log["task_type"]."&".$log["task_objectives"]."&".$log["task_status"]."!";
	}else{
		echo "Database error [Retrieve]!";
	}
}

⭐ Once Nicla Vision requests, update the task status of the latest ongoing food delivery system log.

if(isset($_GET["update_latest_food_system_log_status"])){
	if($db_kiosk_lora_obj->food_delivery_system_log("update", array("task_status" => $_GET["update_latest_food_system_log_status"]))){
		echo "Food delivery system task status is successfully updated!";
	}else{
		echo "Database error [Update]!";
	}
}

⭐ When Nicla Vision transfers an AprilTag sign sample image as a raw RGB565 image buffer:

⭐ Obtain the Apache web server's local time to create the unique image file name.

⭐ Then, save the received raw image buffer (RGB565) as a text (TXT) file to the apriltag_samples folder.

if(!empty($_FILES["captured_image"]['name'])){
	// If exists, obtain the transferred image name.
	$sent_name = (isset($_GET["save_slot_image_sample"])) ? $_GET["save_slot_image_sample"] : "debug_sample_";
	// Get the current date and time.
	$date = date("Y_m_d_H_i_s");
	// Create the sample image file name.
	$img_file = $sent_name.$date;
	
	// Sort the received image file information.
	$captured_image_properties = array(
	    "name" => $_FILES["captured_image"]["name"],
	    "tmp_name" => $_FILES["captured_image"]["tmp_name"],
		"size" => $_FILES["captured_image"]["size"],
		"extension" => pathinfo($_FILES["captured_image"]["name"], PATHINFO_EXTENSION)
	);

    // Check whether the uploaded file extension is in the allowed file formats.
	$allowed_formats = array('jpg', 'png', 'bmp', 'txt');
	if(!in_array($captured_image_properties["extension"], $allowed_formats)){
		echo 'FILE => File Format Not Allowed!';
	}else{
		// Check whether the uploaded file size exceeds the 5 MB data limit.
		if($captured_image_properties["size"] > 5000000){
			echo "FILE => File size cannot exceed 5MB!";
		}else{
			// Save the uploaded file (raw image buffer).
			move_uploaded_file($captured_image_properties["tmp_name"], "./prep_stations/apriltag_samples/".$img_file.".".$captured_image_properties["extension"]);
			echo "FILE => Saved Successfully!";
		}
	}
}

📁 account_config.php

⭐ Initiate the PHP session.

⭐ Include the class.php file to integrate the required functions and define the db_kiosk class object.

session_start();

// Include the required kiosk class functions.
require "class.php";

// Define the db_kiosk class object.
$db_kiosk_obj = new db_kiosk(); 
$db_kiosk_obj->__init__($_db_conn);

⭐ Create a new user (customer) account once the required information is transferred via an HTTP POST request.

⭐ Then, assign the web dashboard status and the given account authentication key to session variables.

if(isset($_POST["user_info"])){
	$response = $db_kiosk_obj->create_new_user_account($_POST["user_info"]);
	echo $response["res"];
	// Change the dashboard status by utilizing a session variable.
	if(str_contains($response["res"], "successfully created")){
		$_SESSION["account"] = "signed";
		$_SESSION["acc_auth_key"] = $response["auth_key"];
	}
}

⭐ Check whether there is a user account with the credentials provided via an HTTP POST request.

⭐ If so, assign the web dashboard status and the retrieved account authentication key to session variables.

if(isset($_POST["login_info"])){
	$response = $db_kiosk_obj->login_to_user_account($_POST["login_info"]);
	echo $response["res"];
	// Change the dashboard status by utilizing a session variable.
	if(str_contains($response["res"], "found successfully")){
		$_SESSION["account"] = "signed";
		$_SESSION["acc_auth_key"] = $response["auth_key"];
	}
}

📁 dashboard_status_update.php

⭐ Include the class.php file to integrate the required functions and define the db_kiosk_status_upt class object.

require "class.php";

// Define the db_kiosk_status_upt class object.
$db_kiosk_status_upt_obj = new db_kiosk_status_upt(); 
$db_kiosk_status_upt_obj->__init__($_db_conn);

⭐ Obtain the status states and logs required by the web application dashboard from the related MariaDB database tables.

if(isset($_GET["get_updates"])){
	echo $db_kiosk_status_upt_obj->generate_dashboard_updates();
}

⭐ If requested, update the enabled (selected) LLM of the current dashboard/kiosk user account.

if(isset($_GET["update_llm"])){
	$db_kiosk_status_upt_obj->update_selected_LLM($_GET["update_llm"]);
}

⭐ If requested, update the deal (menu) preference of the current dashboard/kiosk user account.

if(isset($_GET["update_preference"])){
	$db_kiosk_status_upt_obj->update_deal_preference($_GET["update_preference"]);
}

📁 dashboard_config.js

⭐ Obtain the food menu categories denoting food prep stations.

const available_menu_categories = {
	cat_1: $('div[def="menu_show"]').attr("cat_1"),
	cat_2: $('div[def="menu_show"]').attr("cat_2"),
	cat_3: $('div[def="menu_show"]').attr("cat_3"),
	cat_4: $('div[def="menu_show"]').attr("cat_4"),
	cat_5: $('div[def="menu_show"]').attr("cat_5"),
	cat_6: $('div[def="menu_show"]').attr("cat_6")
};

⭐ Once the user changes the selected large language model (LLM) via the provided drop-down list (HTML select element), update the current dashboard/kiosk user account information automatically via a jQuery Ajax request.

$("#llm_models").on("change", function(event){
	let selected_LLM = $(this).val();
	// Update the user (account) information according to the passed LLM.
	$.ajax({
		url: "/ai_driven_lora_kiosk/assets/dashboard_status_update.php?update_llm=" + selected_LLM,
		type: "GET",
		success: (response) => {
			// Do nothing.
		}
	});	
});

⭐ Once the user changes the deal (menu) preference via the provided HTML radio button group, update the current dashboard/kiosk user account information automatically via a jQuery Ajax request.

$("#deal_preference_change").on("change", function(event){
	let selected_deal_preference = $(this).val();
	// Update the user (account) information according to the passed deal preference.
	$.ajax({
		url: "/ai_driven_lora_kiosk/assets/dashboard_status_update.php?update_preference=" + selected_deal_preference,
		type: "GET",
		success: (response) => {
			// Do nothing.
		}
	});		
});

⭐ In the generate_sign_interface function, generate and return the HTML content of the requested interface — sign-in or sign-up.

function generate_sign_interface(_opt){
	let interface_content = "";
	switch(_opt){
		case "sign_up":
			interface_content = '   <section><label>Firstname: </label><input id="firstname" placeholder="Kutluhan"></input></section> \
									<section><label>Lastname: </label><input id="lastname" placeholder="Aktar"></input></section> \
									<section><label>Email: </label><input id="email" placeholder="contact@kutluhanaktar.com"></input></section> \
									<section><label>Username: </label><input id="username" placeholder="kutluhan123"></input></section> \
									<section><label>Password: </label><input id="password" type="password" placeholder="password1234"></input></section> \
									<section class="radio"> \
									<label>Deal preference: </label> \
									<div> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_1"] + '" checked> ' + available_menu_categories["cat_1"] + ' </label> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_2"] + '"> ' + available_menu_categories["cat_2"] + ' </label> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_3"] + '"> ' + available_menu_categories["cat_3"] + ' </label> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_4"] + '"> ' + available_menu_categories["cat_4"] + ' </label> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_5"] + '"> ' + available_menu_categories["cat_5"] + ' </label> \
									<label> <input type="radio" name="preference" value="' + available_menu_categories["cat_6"] + '"> ' + available_menu_categories["cat_6"] + ' </label> \
									</div> \
									</section> \
									<h2>Add new credit / debit card</h2> \
									<section><label>Card holder name: </label><input id="card_holder_name" placeholder="Kutluhan Aktar"></input></section> \
									<section><label>Card number: </label><input id="card_number" placeholder="378282246310005"></input></section> \
									<section><label>Card expiration date: </label><input type="date" id="card_exp_date"></input></section> \
									<section><label>Card CVV/CVC: </label><input id="card_cvv" placeholder="111"></input></section> \
									<button id="submit_but_sign_up">Submit</button>	';
		break;
		case "sign_in":
			interface_content = '   <section><label>Username: </label><input id="username" placeholder="kutluhan123"></input></section>\
									<section><label>Password: </label><input id="password" type="password" placeholder="password1234"></input></section>\
									<button id="submit_but_sign_in">Login</button> ';
		break;
        default:
			// Do nothing.
        break;		
	}
	return interface_content;
}

⭐ In accordance with the user selection, dynamically update the web dashboard to show the sign-in and sign-up interfaces.

$(".sign_nav > button").on("click", function(event){
	// Check whether the clicked button has already been enabled or not.
	if(!$(this).hasClass("enabled")){
		let selected_interface = $(this).attr("def");
		// Change the account interface HTML content accordingly.
		$(".sign_content").html(generate_sign_interface(selected_interface));
		// Notify the user.
		$(".sign_nav > button").toggleClass("enabled");
	}
});

⭐ Once the user requests, obtain the given user credentials from the sign-in form and make a jQuery Ajax request to log in.

$(".sign_content").on("click", "#submit_but_sign_in", function(event){
	// Check the provided information.
	let login_info = {
		username: $("#username").val(),
		_password: $("#password").val()
	};	
	
	if(login_info["username"] == "" || login_info["_password"] == ""){
		alert("⚠️ Please provide all required fields to login!");
	}else{
		// If the user provided all the required information to login:
		$.ajax({
			url: "/ai_driven_lora_kiosk/assets/account_config.php",
			type: "POST",
			data: {"login_info" : login_info},
			success: (response) => {
				if(response.includes("found successfully")){
					window.location.reload();
				}else{
					alert(response);
				}
			}
		});		
	}
});

⭐ Once the user requests, obtain the provided user information from the sign-up form and check whether all required fields are filled. If so, make a jQuery Ajax request to create a new user account.

$(".sign_content").on("click", "#submit_but_sign_up", function(event){
	// Check the provided information.
	let user_info = {
		firstname: $("#firstname").val(),
		lastname: $("#lastname").val(),
		email: $("#email").val(),
		username: $("#username").val(),
		_password: $("#password").val(),
		menu_preference: $('input[name="preference"]:checked').val(),
		card_holder_name: $("#card_holder_name").val(),
		card_number: $("#card_number").val(),
		card_exp_date: $("#card_exp_date").val(),
		card_cvv: $("#card_cvv").val()
	};
	
	let entered_info_num = Object.keys(user_info).length;
	
	$.each(user_info, (info, val) => {
		if(val == "") entered_info_num--;
	});
	
	if(entered_info_num != 10){
		alert("⚠️ Please provide all required fields to create an account!");
	}else{
		// If the user provided all the required information to create a new account:
		$.ajax({
			url: "/ai_driven_lora_kiosk/assets/account_config.php",
			type: "POST",
			data: {"user_info" : user_info},
			success: (response) => {
				if(response.includes("successfully created")){
					if(confirm(response)){ window.location.reload(); } else{ window.location.reload(); }
				}else{
					alert(response);
				}
			}
		});		
	}
});

⭐ Once requested, log out and discard PHP session parameters by sending an HTTP GET variable.

$('div[def="settings_show"] > section:nth-child(2)').on("click", '.user_info > button[def="logout"]', () => {
	if(!window.location.href.includes("?")) window.location.href += "?logout";
});

⭐ When the user requests to generate user-specific menus/deals, open the large language model (LLM) interface, which triggers the generate_LLM_based_user_specific_deals.php file by embedding it into an HTML iframe element.

#️⃣ By applying the LLM interface as a nested browsing context, I was able to make the restaurant web application dashboard behave as a single-page application.

$('div[def="settings_show"] > section:nth-child(2)').on("click", '.user_info > button[def="generate_new_deals"]', () => {
	let llm_interface = $(".LLM_interface");
	llm_interface.children("iframe").attr("src", "generate_LLM_based_user_specific_deals.php");
	if(!llm_interface.hasClass("enabled")) llm_interface.addClass("enabled");
});

⭐ When the user requests, close the LLM interface and remove the HTML iframe element content.

$(".LLM_interface").on("click", function(event){
	if($(this).hasClass("enabled")) $(this).removeClass("enabled");
	$(this).children("iframe").attr("src", "");
	window.location.reload();
});

📁 dashboard_status_update.js

⭐ Every 2 seconds, make a jQuery Ajax request to obtain the necessary status states, logs, and task information from the MariaDB database as a JSON object.

⭐ Then, decode the retrieved JSON object to update the associated HTML elements of the web dashboard accordingly so as to inform the current dashboard/kiosk user (customer) of ongoing restaurant operations.

setInterval(() => {
	// Obtain the required updates from the database.
	$.ajax({
		url: "/ai_driven_lora_kiosk/assets/dashboard_status_update.php?get_updates",
		type: "GET",
		success: (response) => {
				// After getting the necessary dashboard updates, validate retrieved information to show the updates accurately.
				let upt_obj = JSON.parse(response);
				// Kiosk status.
				if(upt_obj["kiosk_status"] != "not_found"){
					let kiosk_status = (upt_obj["kiosk_status"] == "authorized") ? "Authorized by Customer" : "Not Authorized";
					$("#current_kiosk_status").text(kiosk_status);
				}else{
					$("#current_kiosk_status").text("⛔");
				}
				// Activated LLM.
				if(upt_obj["enabled_LLM"] != "not_found"){
					$("#current_activated_llm").text(upt_obj["enabled_LLM"]);
				}else{
					$("#current_activated_llm").text("⛔");
				}
				// Latest order log.
				if(upt_obj["latest_order_log_info"] != "not_found"){
					$("#current_order_type").text((upt_obj["latest_order_log_info"]["order_type"] == "order_generic") ? "Generic Deal" : "User-specific Deal");
					$("#current_order_tag").text(upt_obj["latest_order_log_info"]["order_tag"]);
					$("#current_order_status").text(upt_obj["latest_order_log_info"]["order_status"]);
					$("#current_order_upt_time").text(upt_obj["latest_order_log_info"]["order_upt_time"]);
				}else{
					$("#current_order_type").text("⛔");
					$("#current_order_tag").text("⛔");
					$("#current_order_status").text("⛔");
					$("#current_order_upt_time").text("⛔");
				}
				// Latest LoRa task log.
				if(upt_obj["latest_lora_log_info"] != "not_found"){
					$("#current_lora_code").text(upt_obj["latest_lora_log_info"]["lora_code"]);
					$("#current_lora_type").text(upt_obj["latest_lora_log_info"]["task_type"]);
					$("#current_lora_details").text(upt_obj["latest_lora_log_info"]["task_details"]);
					$("#current_lora_status").text(upt_obj["latest_lora_log_info"]["task_status"]);
					$("#current_lora_device_id").text(upt_obj["latest_lora_log_info"]["device_id"]);
					$("#current_lora_gateway_id").text(upt_obj["latest_lora_log_info"]["gateway_id"]);
					$("#current_lora_upt_time").text(upt_obj["latest_lora_log_info"]["up_date"]);
				}else{
					$("#current_lora_code").text("⛔");
					$("#current_lora_type").text("⛔");
					$("#current_lora_details").text("⛔");
					$("#current_lora_status").text("⛔");
					$("#current_lora_device_id").text("⛔");
					$("#current_lora_gateway_id").text("⛔");
					$("#current_lora_upt_time").text("⛔");
				}			
			}
		});	
}, 2000);

📁 Index.php

This is the home page of the restaurant web application, combining all of the previous files to provide a smooth single-page application experience for customers using the web dashboard. Please refer to the project GitHub page to review this file fully.

⭐ Include the class.php file to integrate the required functions and define the db_kiosk_llm class object.

⭐ Initiate the PHP session.

require "assets/class.php";

// Define the db_kiosk_llm class object.
$db_kiosk_llm_obj = new db_kiosk_llm(); 
$db_kiosk_llm_obj->__init__($_db_conn);

// Start the session.
session_start();

⭐ When the user requests to log out, discard PHP session parameters and remove the current dashboard/kiosk user information from the MariaDB database.

⭐ Then, reload the web dashboard without the passed HTTP GET parameters to avoid errors.

if(isset($_GET["logout"])){
	session_unset();
	session_destroy();
	// Remove the current dashboard/kiosk user information.
	$db_kiosk_llm_obj->discard_current_kiosk_user();
	// Reload the page without the passed GET parameters to avoid errors.
	header("Location: /ai_driven_lora_kiosk");
	exit();
}

⭐ If the user logs in successfully, fetch the associated account information from the MariaDB database.

⭐ Obtain the food menu categories denoting food prep stations.

$dash_status = (isset($_SESSION["account"]) && $_SESSION["account"] == "signed") ? true : false;

// If signed, get the registered user information.
$user_info = ($dash_status) ? $db_kiosk_llm_obj->obtain_user_information($_SESSION["acc_auth_key"]) : "none";

// Obtain the available menu categories by prep station.
$menu_categories = $db_kiosk_llm_obj->obtain_available_food_item_info("category");

⭐ Obtain 12 different generic menus/deals from the database and process the retrieved information to generate HTML content for each menu dynamically.

$generic_menu_information = $db_kiosk_llm_obj->obtain_menu_deal_information("generic", "all");
if($generic_menu_information != false){
	$menu_num = 0;
	foreach($generic_menu_information as $menu){
		$menu_num++;
		// Get the food item list for the given menu / deal.
		$food_item_list = '';
		foreach($menu["item_list"] as $item){
			$food_item_list .= '<button>'.$item["name"].' <span> x '.$item["amount"].'</span></button>';
		}
		// Show each generic menu / deal.
		echo '
				<div>
				<img src="assets/img/generic_menu_'.$menu_num.'.png" />
				<article>
				<h2 def="order_code">'.$menu["order_tag"].'</h2>
				<p def="menu_definition">'.$menu["definition"].'</p>
				'.$food_item_list.'
				<p def="menu_discount">Discount: <span def="d_percentage">'.$menu["discount_percentage"].'%</span></p>
				<p def="menu_price">Price: <span def="t_price">$'.$menu["total_price"].'</span> <span def="d_price">$'.$menu["discounted_price"].'</span></p>
				</article>
				</div>		
	         ';
	}
}else{
	echo "<h2>Database error!</h2>";
}

⭐ If the customer logged in successfully and generated user-specific menus/deals via an available large language model (LLM), obtain 12 different user-specific menus/deals from the database and process the retrieved information to create HTML content for each menu dynamically.

⭐ Otherwise, notify the customer of the steps required to generate user-specific menus/deals via the available LLMs.

if($dash_status == false){
	echo '
			<div>
			<img src="assets/img/specific_menu_not_signed.png" />
			<article>
			<p def="menu_definition">🚫 Please sign in to enable the customer dashboard to generate your user-specific menus / deals!</p>
			</article>
			</div>		
	     ';
}else{
	// Obtain user-specific menu / deal information from the database, that of the current user account if signed in successfully.
	$user_specific_menu_information = $db_kiosk_llm_obj->obtain_menu_deal_information($_SESSION["acc_auth_key"], "all");
	if($user_specific_menu_information != false){
		if($user_specific_menu_information[0] == NULL){
			echo '
					<div>
					<img src="assets/img/specific_menu_pending.png" />
					<article>
					<p def="menu_definition">⚠️ Please utilize the customer dashboard to select an available LLM and generate your user-specific menus / deals!</p>
					</article>
					</div>		
				 ';			
		}else{
			// Show each user-specific menu / deal if the user employed an available LLM to produce menus / deals based on the selected deal preference.
			$menu_num = 0;
			foreach($user_specific_menu_information as $menu){
				$menu_num++;
				// Get the food item list for the given menu / deal.
				$food_item_list = '';
				foreach($menu["item_list"] as $item){
					$food_item_list .= '<button>'.$item["name"].' <span> x '.$item["amount"].'</span></button>';
				}
				// Show.
				echo '
						<div>
						<img src="assets/img/specific_menu_'.$menu_num.'.png" />
						<article>
						<h2 def="order_code">'.$menu["order_tag"].'</h2>
						<p def="menu_definition">'.$menu["definition"].'</p>
						'.$food_item_list.'
						<p def="menu_discount">Discount: <span def="d_percentage">'.$menu["discount_percentage"].'%</span></p>
						<p def="menu_price">Price: <span def="t_price">$'.$menu["total_price"].'</span> <span def="d_price">$'.$menu["discounted_price"].'</span></p>
						</article>
						</div>		
					 ';			
				}
		}
	}else{
		echo "<h2>Database error!</h2>";
	}
}

⭐ Create the drop-down list (HTML select element) for the available large language models (LLMs).

⭐ If the customer logged in successfully, show the highlighted LLM option on the drop-down list as the enabled (activated) LLM of the given customer account by adding the selected attribute.

⭐ Otherwise, disable user interaction for the LLM drop-down list.

$available_LLMs = ["deepseek-r1:8b", "deepseek-r1:7b", "deepseek-r1:1.5b", "gemma3:4b", "gemma3:1b", "llama3.2:3b", "qwen3:4b", "phi4-mini"];
$llm_change_disable = ($dash_status) ? "" : "disabled";
$llm_option_html_content = '<select name="llm_models" id="llm_models" '.$llm_change_disable.'>';
foreach($available_LLMs as $llm){
	if($dash_status && $llm == $user_info["activated_LLM"]){
		$llm_option_html_content .= '<option value="'.$llm.'" selected>'.$llm.'</option>';
	}else{
		$llm_option_html_content .= '<option value="'.$llm.'">'.$llm.'</option>';
	}
}
$llm_option_html_content .= '</select>';
echo $llm_option_html_content;

⭐ According to the user account state, change the HTML content of the web dashboard interface automatically and dynamically.

if($dash_status){
	// Generate the deal preference options based on the given menu categories.
	$deal_preference_select = '<select name="deal_preference_change" id="deal_preference_change">';
	foreach($menu_categories as $cat){
		if($cat == $user_info["menu_preference"]){
			$deal_preference_select .= '<option value="'.$cat.'" selected>'.$cat.'</option>';
		}else{
			$deal_preference_select .= '<option value="'.$cat.'">'.$cat.'</option>';
		}
	}
	$deal_preference_select .= "</select>";
	// Show the necessary account and ongoing task information.
	$dashboard_acc_content = '<div class="user_info">
							  <h2>Invoice</h2>
							  <section><p>Customer Name:</p><span>'.$user_info["firstname"].' '.$user_info["lastname"].'</span></section>
							  <section><p>Customer Email:</p><span>'.$user_info["email"].'</span></section>
							  <section><p>Registered Card:</p><span>'.$user_info["card_number"].'</span></section>
							  <section><p>Card Holder Name:</p><span>'.$user_info["card_holder_name"].'</span></section>
							  <section><p>Card Exp. Date:</p><span>'.$user_info["card_exp_date"].'</span></section>
							  <h2>User-specific Deals</h2>
							  <section><p>Enabled LLM:</p><span id="current_activated_llm">'.$user_info["activated_LLM"].'</span></section>
							  <section><p>Deal Preference:</p>'.$deal_preference_select.'</section>
							  <button def="generate_new_deals">Generate New <span>LLM</span> Deals</button>							  
							  <h2>Track Order</h2>
							  <section><p>Kiosk Status:</p><span id="current_kiosk_status">🔄</span></section>
							  <section><p>Order Type:</p><span id="current_order_type">🔄</span></section>
							  <section><p>Order Tag:</p><span id="current_order_tag">🔄</span></section>
							  <section><p>Status:</p><span id="current_order_status">🔄</span></section>
							  <section><p>Updated:</p><span id="current_order_upt_time">🔄</span></section>
                              <h2>Review LoRa Task</h2>
							  <section><p>Code:</p><span id="current_lora_code">🔄</span></section>							  
							  <section><p>Type:</p><span id="current_lora_type">🔄</span></section>							  
							  <section><p>Details:</p><textarea id="current_lora_details">🔄</textarea></section>							  
							  <section><p>Status:</p><textarea id="current_lora_status">🔄</textarea></section>							  
							  <section><p>Device ID:</p><span id="current_lora_device_id">🔄</span></section>
							  <section><p>Gateway ID:</p><span id="current_lora_gateway_id">🔄</span></section>
							  <section><p>Requested:</p><span id="current_lora_upt_time">🔄</span></section>  
							  <h2>Account</h2>
							  <section><p>Username:</p><span>'.$user_info["username"].'</span></section>
							  <section><p>Authentication Code:</p><span>'.$user_info["authentication_key"].'</span></section>
							  <button def="logout">Logout</button>
                              </div>';
}else{
	// Show the sign in form.
	$dashboard_acc_content = '<div class="sign_nav">
							  <button class="enabled" def="sign_in">Sign In</button>
							  <button def="sign_up">Sign Up</button>
							  </div>
							  <div class="sign_content">
							  <section><label>Username: </label><input id="username" placeholder="kutluhan123"></input></section>
							  <section><label>Password: </label><input id="password" type="password" placeholder="password1234"></input></section>
							  <button id="submit_but_sign_in">Login</button>
							  </div>';
}
echo $dashboard_acc_content;

📁 dashboard_style_root_settings.css and dashboard_style.css

Please refer to the project GitHub page to review the restaurant web dashboard design (styling) files.

📁 order_schedule_update.php

⭐ Include the class.php file to integrate the required functions and define the db_kiosk_status_order class object.

require "class.php";

// Define the db_kiosk_status_order class object.
$db_kiosk_status_order_obj = new db_kiosk_status_order(); 
$db_kiosk_status_order_obj->__init__($_db_conn);

⭐ Produce the HTML table content from the registered food delivery system logs to inform drive-through restaurant workers operating the food prep stations of the latest customer orders placed for generic or user-specific menus/deals, showcasing food items required from each prep station thoroughly.

$obtained_schedule_updates = $db_kiosk_status_order_obj->generate_order_schedule_updates();

$schedule_updates = '<tr><th>Account Authentication Key</th><th>Menu Tag</th><th>Deal Type</th><th>Station 1</th><th>Station 2</th><th>Station 3</th><th>Station 4</th><th>Station 5</th><th>Station 6</th><th>Order Status</th><th>Updated</th></tr>';

if($obtained_schedule_updates == false){
	$schedule_updates .= '<tr class="none"><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td><td>none</td></tr>';
}else{
	$schedule_updates .= $obtained_schedule_updates;
}

// Print the produced HTML table content.
echo $schedule_updates;

📁 order_schedule_update.js

⭐ Every 2 seconds, make a jQuery Ajax request to obtain the latest placed order information as HTML table rows from the MariaDB database.

⭐ Then, update the associated HTML elements accordingly to inform drive-through restaurant workers of the latest customer orders placed for generic or user-specific menus/deals.

setInterval(() => {
	// Obtain the required updates from the database.
	$.ajax({
		url: "/ai_driven_lora_kiosk/assets/order_schedule_update.php",
		type: "GET",
		success: (response) => {
				// After getting the necessary order schedule updates, append them into the given HTML table.
                $('.order_schedule > table').html(response);			
			}
		});	
}, 2000);

📁 order_schedule.php and order_schedule.css

Please refer to the project GitHub page to inspect the order schedule home page and design (styling) files.

📁 RGB565_raw_buffer_converter.py

Since I did not need to convert the received raw RGB565 image buffers to JPG files automatically via the web application, I programmed a simple Python script to convert all of the raw image buffers saved as text (TXT) files manually.

⭐ Include the required modules.

from glob import glob
import numpy as np
from PIL import Image
import os

⭐ Obtain all of the RGB565 raw image buffers transferred by Arduino Nicla Vision as text (TXT) files.

⭐ Convert each retrieved RGB565 buffer (text file) to a JPG image file and save the produced image files with the original file names to the jpg_converted folder.

⭐ After saving the produced JPG image files successfully, remove the converted RGB565 raw image buffers (text files) from the web server.

path = "../apriltag_samples"
raw_buffers = glob(path + "/*.txt")

# Convert each retrieved RGB565 raw image buffer to a JPG image and save the generated image files to the jpg_converted folder.
for buf in raw_buffers:
    # Define the required image information.
    loc = path + "/jpg_converted/" + buf.split("/")[-1].split(".")[0] + ".jpg"
    size = (320,320)
    # Conversion: RGB565 (uint16_t) to RGB (3x8-bit pixels, true color)
    raw = np.fromfile(buf).byteswap(True)
    file = Image.frombytes('RGB', size, raw, 'raw', 'BGR;16', 0, 1)
    # Save the generated JPG image file.
    file.save(loc)
    print("Converted: " + loc)
    # After converting and saving the JPG image file successfully, remove the converted RGB565 raw image buffer (.txt).
    os.remove(buf);

Step 11: Running the PHP-MQTT client operated by the web application on LattePanda Mu

After concluding programming my PHP-MQTT client, operated by the restaurant web application, to connect to The Things Stack MQTT server (broker), I needed to run the PHP-MQTT client continuously to make sure the web application receives each LoRa-transmitted data packet from the kiosk customer endpoint without interruptions.

In this regard, after initiating the Apache web server on LattePanda Mu, I executed the PHP-MQTT client separately via the terminal. I chose to execute all subsystems manually to pinpoint errors easily while extensively testing the web application features. Nevertheless, it is possible to create a cron job to automate all subsystem initiations, including the PHP-MQTT client.

Please refer to the previous step to inspect the code file of the PHP-MQTT client — ttn_mqtt_client.php.

#️⃣ First, I navigated to the restaurant web application root folder on the terminal.

cd /opt/lampp/htdocs/ai_driven_lora_kiosk/

#️⃣ Then, I executed the PHP-MQTT client via the terminal.

php ttn_mqtt_client.php

#️⃣ As mentioned in the code documentation (previous step), the PHP-MQTT client receives LoRa-transmitted uplink messages from The Things Stack MQTT server by subscribing to the associated MQTT topic, processes the received data packets to perform the requested tasks, and registers LoRa task (transmission) logs to the associated MariaDB database table.

Step 12: Fine-tuning system prompts and objectives (inputs) for LLMs operated by the web application to generate reliable user-specific menus/deals

After integrating the Ollama-PHP helper library and programming the restaurant web application to execute the available large language models (LLMs), I started to experiment with system prompts and objectives (inputs) to generate user-specific menus/deals based on the provided customer deal preference.

Although I meticulously examined the performance of Ollama's various large language models with basic objectives via the terminal to determine the available LLMs for the restaurant web application, I waited to fine-tune and optimize system prompts and objectives until I configured the Ollama-PHP helper library to review the LLM responses on the terminal and the web application simultaneously.

#️⃣ After my initial tests, I noticed that thinking models have a tendency to generate inconsistent results or hallucinations for my straightforward restaurant menu generation use case since they gaslight themselves to tangential outcomes instead of utilizing predefined food item information.

#️⃣ In this regard, as explained in previous steps, I disabled thinking via the built-in Ollama chat API while executing the available LLMs via the web application.

#️⃣ After disabling thinking, I proceeded to fine-tune system prompts and objectives (inputs) to ensure each available LLM generates reliable and valid user-specific restaurant menus/deals in the passed JSON format.

#️⃣ After extensively testing different iterations of system prompts and objectives, I obtained the most reliable results with these configurations below. Please refer to the previous steps to review the code implementation.

💡 System prompts

'Answer as a drive-through restaurant kiosk computer.'

💡 Objectives (inputs)

'
Generate 12 different menus using food items from these JSON objects:
								
'.$available_food_items_and_categories.'
								
Each menu has to include at least one item from the category '.$customer_deal_preference.'.
You must add additional items from the category Side Dishes and Desserts to each menu.
Each menu has to include up to 5 different items.
You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category '.$customer_deal_preference.'.
You must calculate the total menu price by using the provided food item prices and the assigned amounts.
You must assign a discount percentage from 2 percent to 8 percent for each menu.
You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage.
You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words.
You must assign an order tag for each of the 12 different menus, from e001 to e012.
								
You must generate and return 12 different menus and return them as JSON objects in this format:				
								
'.$generated_menu_json_format;

After months of developing this research project, I can certainly state that the best-performing large language model (LLM) rankings for my use case are as follows:

• deepseek-r1:8b
• qwen3:4b
• gemma3:4b
• deepseek-r1:7b
• llama3.2:3b
• phi4-mini
• deepseek-r1:1.5b
• gemma3:1b

Not surprisingly, due to their parameter counts, deepseek-r1:1.5b and gemma3:1b are definitely the worst-performing models. Even if they rarely managed to produce menus with all of the required information, they repeatedly generated faulty JSON objects. Since this is a proof-of-concept research project, I did not remove them from the available LLMs and presented their inaccurate results in the following steps to showcase how malfunctioning LLMs can detrimentally impact the customer experience and an AI-powered drive-through restaurant workflow.

Step 13: Prototyping the kiosk customer endpoint PCB and testing its electronic components

To build a multifunctional LoRa-enabled kiosk customer endpoint, I modified, tested, and analyzed electrical components until I managed to accomplish all of the features I wanted to implement, considering a market-ready drive-through restaurant establishment.

#️⃣ As mentioned earlier, RA-08H LoRaWAN node board comes with an integrated Raspberry Pi RP2040 and provides six Crowtail interfaces for different sensors, as well as a 20-pin header for meeting further development requirements.

#️⃣ First, I extrapolated the detailed Crowtail interface and 20-pin header pinouts by inspecting the provided node board schematics on KiCad.

#️⃣ Then, I soldered the 20-pin header to RA-08H node board.

#️⃣ Since I needed a reliable and cost-effective camera for developing vehicle detection features, I decided to utilize a Crowtail TTL serial camera. This module provides NTSC video output and is actually for surveillance systems. Nonetheless, thanks to its built-in UART features, the module can act as a JPEG color camera and save a snapshot from the video stream to its internal buffer to transfer it via serial communication.

#️⃣ After experimenting with the TTL camera module's NTSC video output by connecting a spare RCA cable to the associated pins, I decided to discard the NTSC connection since it was too janky for an AI-powered drive-through restaurant :)

#️⃣ To show the kiosk endpoint interface to customers, I decided to utilize a Fermion 2.0" TFT LCD. Since the Fermion display provides an integrated microSD card module, I was able to save the collected vehicle image samples effortlessly.

#️⃣ To enable customers to interact with the kiosk endpoint interface, I designed a special 4x4 matrix keypad. I also added an RGB LED to notify customers.

#️⃣ Since I wanted to supply the kiosk customer endpoint via my old powerbank, I simply utilized a USB buck-boost converter board.

#️⃣ As depicted below, I made all component connections according to the available and compatible pins of the onboard RP2040 of RA-08H LoRaWAN node board.

// Connections
// RA-08H LoRaWan Node Board w/ RP2040 :
//                                Fermion 2.0" IPS TFT LCD Display (320x240)
// +5V     ------------------------ V
// GND     ------------------------ G
// 18/SCK  ------------------------ CK
// 19/MOSI ------------------------ SI
// 16/MISO ------------------------ SO
// 17/CS   ------------------------ CS
// 20      ------------------------ RT
// 21      ------------------------ DC
// -       ------------------------ BL
// 15      ------------------------ SC
//                                Crowtail - Serial Camera
// +5V     ------------------------ 5V
// GND     ------------------------ GND
// 8/TX1_H ------------------------ RX
// 9/RX1_H ------------------------ TX
//                                Custom KeyPad (4x4)
// 6       ------------------------ R1
// 7       ------------------------ R2
// 10      ------------------------ R3
// 11      ------------------------ R4
// 22      ------------------------ C1
// 23      ------------------------ C2
// 24      ------------------------ C3
// 25      ------------------------ C4
//                                5mm Common Anode RGB LED
// 12      ------------------------ R
// 13      ------------------------ G
// 14      ------------------------ B

Step 13.1: Designing the kiosk customer endpoint PCB outline, protective case, and layout

After ensuring the electrical components of the kiosk customer endpoint perform as expected, I started to work on the kiosk endpoint PCB layout. After developing a lot of PCBs for my proof-of-concept projects, I came to the conclusion that designing PCB outlines and structures (silkscreen, copper layers, etc.) directly on Autodesk Fusion 360 works best for my development process. Creating PCB digital twins allows me to simulate complex 3D mechanical systems compatible with the PCB part placement and outline before sending my PCB designs for manufacturing.

As I was designing this PCB, I wanted to add Cookie Monster as a part of the PCB outline since who could be more welcoming to a drive-through restaurant than this renowned Muppet :)

While designing the snug-fit PCB case, I noticed that the PCB size would be too large since I wanted to protrude the custom 4x4 keypad section. Thus, I decided to place some components on the back of the PCB, letting me design a compact and robust kiosk customer endpoint.

In this regard, I needed to leave some areas on the back of the PCB case open to place the associated electrical components. Thus, I decided to design special threaded bolts and nuts (3D-printable) to securely attach the kiosk customer endpoint PCB to its protective case.

After designing the PCB outline and structure, I imported my outline graphic to Kicad 8.0 in the DXF format and created the necessary circuit connections to complete the kiosk customer endpoint PCB layout.

Step 13.2: Soldering and assembling the kiosk customer endpoint PCB

After completing the kiosk customer endpoint PCB layout, I utilized ELECROW's high-quality regular PCB (Rigid) manufacturing service. For further inspection, I provided the fabrication files of this PCB on the project GitHub page. To replicate this device, you can order this PCB directly from my ELECROW community page.

#️⃣ After receiving my PCBs, I soldered electronic components and pin headers via my TS100 soldering iron to place all parts according to my PCB layout.

📌 Component assignments on the kiosk customer endpoint PCB:

RA_08H_Board_1 (Headers for RA-08H LoRaWan Node Board)

RA_08H_Port_1 (Headers for RA-08H Crowtail UART Port)

Crowtail_Cam_1 (Headers for Crowtail Serial Camera)

NTSC_1 (Headers for Crowtail - Serial Camera NTSC Pins)

Fermion_TFT_Display_1 (Headers for Fermion: 2.0" TFT LCD Display)

K1, K2, K3, K4, K5, K6, K7, K8, K9, K10, K11, K12, K13, K14, K15, K16 (6x6 Pushbutton)

D1 (5 mm Common Anode RGB LED)

J_5V_1 (DC Barrel Female Power Jack)

J_5V_2 (Headers for Power Supply)

Step 14: Prototyping the food delivery system Flex PCB and testing its electronic components

To build a full-fledged automatic food delivery system, I modified, tested, and analyzed electrical components until I managed to accomplish all of the features I wanted to implement, including establishing the data transfer process with the restaurant web application.

As mentioned earlier, Arduino Nicla Vision is a compact IoT (Wi-Fi-enabled) development board providing an integrated 2MP color camera (GC2145).

#️⃣ First, I inspected Nicla Vision's pinout to determine available GPIOs.

#️⃣ Since I decided to base my automatic food delivery system on the H-Bot gantry mechanism, I needed to utilize three Nema 17 (17HS3401) stepper motors to enable gantry head movements on the Cartesian axes, driven by a single GT2 timing belt, and rotate the food carrier arm (tray). Please refer to the following steps to review my H-Bot-inspired structure.

#️⃣ To control Nema 17 stepper motors with Nicla Vision, I employed three A4988 driver modules.

#️⃣ Since I wanted to enable fully automated homing sequences on the axes for the gantry head, I added two limit (micro) switches — KW10-Z2P.

#️⃣ Although I developed the food delivery system to be controlled by the web application remotely, I still needed to test stepper motor movements for debugging and capture AprilTag sign image samples manually. Thus, I connected four control buttons to Nicla Vision.

#️⃣ Since I needed to supply power for a lot of current-demanding electronic components with different operating voltages, I decided to convert my old ATX power supply unit (PSU) into a simple bench power supply by utilizing an ATX adapter board (XH-M229) providing stable 3.3V, 5V, and 12V. For each power output of the adapter board, I soldered wires to attach a DC-barrel-to-wire jack (male) in order to create a production-ready bench power supply.

#️⃣ As depicted below, I made all component connections according to the available and compatible pins of Nicla Vision.

// Connections
// Arduino Nicla Vision :  
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 1]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// PE_12   ------------------------ DIR
// PE_13   ------------------------ STEP
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 2]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// PE_14   ------------------------ DIR
// PE_11   ------------------------ STEP
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 3]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// PG_12   ------------------------ DIR
// PB_9    ------------------------ STEP
//                                Control Button (A)
// PC_4    ------------------------ +
//                                Control Button (B)
// PF_13   ------------------------ +
//                                Control Button (C)
// PF_3    ------------------------ +
//                                Control Button (D)
// PB_8    ------------------------ +
//                                Micro Switch [A]
// PA_9    ------------------------ +
//                                Micro Switch [B]
// PA_10   ------------------------ +

Step 14.1: Designing the food delivery system Flex PCB outline and pliable case with Fusion 360 Sheet Metal tools

After ensuring the electrical components of the automatic food delivery system perform as intended, I started to work on the food delivery system PCB layout. However, the PCB design process was quite different since I wanted to build the gantry head around a single PCB without any additional wiring other than stepper motor connections. Due to my H-Bot-inspired structure and the food prep stations' orientation, Nicla Vision (built-in camera) and limit switches must be placed perpendicular to the top surface of the ganty head at different positions — front, left, and back.

As a single Rigid PCB, this PCB layout would not be possible. Therefore, I decided to design a Flex PCB meeting my requirements for the gantry head. Flexible printed circuits (FPCs) most commonly utilize Polyimide (PI) as the thermal insulating material, supplied with a thin polymer coating to protect the affixed conductive circuit patterns. The bend radius of Flex PCBs depends on the number of layers and board thickness.

As explained earlier, according to my PCB development process, I designed the Flex PCB outline on Autodesk Fusion 360 as compact as possible, considering component dimensions and the required bending areas to place parts perpendicular to the top surface. Then, I designed a snug-fit case for the Flex PCB.

However, contrary to my Rigid PCB case structures, I needed to make my Flex PCB case structure conform to the bending areas and absorb impact forces while performing the ganty head homing sequences. Otherwise, since Flex PCB through-hole pads are brittle and susceptible to warping, the Flex PCB would easily get damaged while moving the gantry head on the axes.

In this regard, I needed to simulate the bent state of the Flex PCB case to ensure the bending areas conform to the case curves. Even though the parts should be placed perpendicular, I could not apply 90-degree bends to the Flex PCB since tighter bends increase circuit damage exponentially. Therefore, I calculated the required distance to place parts with 45-degree bends (gradual) and modified the bending area spans of the case accordingly.

Fusion 360 does not have tools to adjust the ductility of solid 3D models directly. Nonetheless, it is possible to utilize the built-in sheet metal tools to bend solid 3D models as needed.

#️⃣ First, I navigated to the Sheet Metal section and created a Flange from the projection of the bottom surface of the Flex PCB case.

#️⃣ Then, I determined the bend lines at the middle of the bending area spans and applied bends with a suitable bend radius.

#️⃣ I reduced the sheet metal thickness to 0.5 mm and unfolded the bends to construct a thin and pliable surface.

#️⃣ Then, I combined the pliable sheet metal surface with the Flex PCB case — solid 3D model. Since the Flex PCB case carried the sheet metal attributes after combining, I was able to refold the bends to simulate the bent (folded) state of the case.

#️⃣ Thanks to the parametric timeline, I was able to access the unfolded version of the Flex PCB case for printing it by utilizing TPU without supports.

Step 14.2: Designing the food delivery system Flex PCB layout and adding stiffeners to enhance mechanical strength

Even though I designed the pliable case (printed with TPU) to reduce impact forces, I still needed to reinforce the opposite side of the welding surface of the Flex PCB (top) to solder THT (through-hole) components without damaging pads or traces.

Instead of employing the more complex and expensive Rigid-Flex PCB manufacturing technique, I decided to provide high mechanical strength and support to the areas populated with THT components by merely adding FR4 stiffeners.

In this regard, before proceeding with constructing the Flex PCB layout, I designed the stiffener extents on Fusion 360.

After designing the Flex PCB outline, structure, and stiffener areas, I imported my outline and stiffener graphics to Kicad 8.0 in the DXF format. Then, I created the necessary circuit connections to complete the food delivery system Flex PCB layout.

As I was constructing my FPC stackup, I decided to represent FR4 stiffeners via the Margin layer.

According to the gantry head dimension requirements, I assigned the Flex PCB fabrication parameters as follows.

• PCB Layer: 2-Layer
• Board Thickness: 0.1 mm
• Stiffener Material: FR-4
• Stiffener Thickness: 1.6 mm
• Copper Weight: 0.5 oz

Step 14.3: Soldering and assembling the food delivery system Flex PCB

After completing the food delivery system Flex PCB layout, I utilized ELECROW's reliable premium PCB (Flex) manufacturing service. For further inspection, I provided the fabrication files of this PCB on the project GitHub page.

Nonetheless, Flex PCB fabrication is not a straightforward process like the regular (Rigid) PCB service. So, please refer to the manufacturing guidelines to order this PCB to replicate the gantry head.

After receiving my Flex PCBs (with stiffeners), I deliberately soldered electronic components and pin headers via my TS100 soldering iron to place all parts according to my PCB layout.

📌 Component assignments on the food delivery system Flex PCB:

Nicla_Vision1 (Headers for Arduino Nicla Vision)

Motor1, Motor2, Motor3 (Headers for Nema 17 [17HS3401] Stepper Motor)

DR1, DR2, DR3 (A4988 Stepper Motor Driver)

SW1, SW2 (KW10-Z2P Micro Switch)

C1, C2, C3, C4 (6x6 Pushbutton)

J_5V_1, J_12V_1 (DC Barrel Female Power Jack)

J_5V_2, J_12V_2 (Headers for Power Supply)

Step 15: Developing custom parts, components, and mechanisms to build a full-fledged drive-through kiosk and food delivery system

In accordance with the discussed AI-assisted features, I designed a drive-through restaurant establishment from the ground up, including the kiosk customer endpoint, the vehicle platform, the H-Bot-inspired automatic food delivery system, and more.

I utilized Autodesk Fusion 360 as my main CAD software to design all of the drive-through restaurant establishment components, including the H-Bot-inspired gantry head mechanical parts.

As I was designing 3D parts, I leveraged some third-party CAD files to obtain accurate measurements to build a flawless restaurant establishment.

✒️ Nema 17 (17HS3401) Stepper Motor (Step) | Inspect

✒️ LattePanda Mu Full-Function Evaluation Carrier Board (Step) | Inspect

✒️ Arduino Nicla Vision (Step) | Inspect

✒️ Raspberry Pi 4 Model B (Step) | Inspect

Considering AI-powered drive-through restaurant features, my establishment layout consists of four subsystems exercising different operations relating to customer relations, special menu/deal generation, kiosk/web dashboard interactions, order management, food preparation, and food service.

In order to build distinct restaurant subsystems, I utilized a wide range of PLA filaments with various attributes for each subsystem. By combining my filament selection with multi-color printing, I was able to produce high-quality and appealing components.

As mentioned earlier, I printed the food delivery system Flex PCB case with a TPU 95A filament to provide elasticity and resilience.

🎨 For the kiosk customer endpoint components:

• ePLA-Silk Mystic Gold-Red-Green
• PLA+ Yellow

🎨 For the food prep station components:

• CR-Silk Yellow-Blue
• ePLA-Matte Deep Black
• ePLA-Matte Milky White
• PLA+ Beige
• PLA+ Yellow
• PLA+ Peak Green
• PLA+ Fire Engine Red

🎨 For the H-Bot-based food delivery system components:

• eTPU-95A White
• Hyper PLA Spring Lake
• Hyper PLA Peach Fuzz
• ePLA-Matte Milky White

🎨 For the restaurant operation control panel components:

• CR-Silk Gold-Red
• ePLA-Matte Deep Black
• ePLA-Matte Milky White

Since I needed to print some components in multi-color and achieve high precision with TPU for the Flex PCB case, I decided to utilize Bambu Lab A1 Combo. I enjoyed the easy integration process of AMS lite and was able to slice my models effortlessly with Bambu Lab's official slicer, Bambu Studio, since it is based on PrusaSlicer.

In the following steps, I will explain my design and assembly process for each restaurant subsystem in detail. As a proof-of-concept project, I wanted to make my restaurant establishment layout open-source. So, I made all the design (STL) files available on the project GitHub page.

Step 15.a: Designing the drive-through kiosk customer endpoint and the vehicle platform

#️⃣ As I had already designed the kiosk customer endpoint PCB case, I started to work on its mount stand.

#️⃣ To balance the weight distribution of the customer endpoint, I added snug-fit holes for four 20 mm steel balls (beads) at the stand base.

#️⃣ I placed matching M3 holes on the PCB case and its stand to connect them via compatible M3 screws and nuts.

#️⃣ Even though I had designed the special threaded bolts and nuts for attaching the customer endpoint PCB to its case, adding the stand engendered clearance issues for the bottom nuts. Thus, I designed longer versions of the special threaded bolts.

#️⃣ Finally, I designed the vehicle platform, providing a horizontal window to access the food carrier arm (tray).

Step 15.a.1: Printing the custom parts of the kiosk customer endpoint, the vehicle platform, and the customer endpoint PCB case

#️⃣ First, on Autodesk Fusion 360, I exported all kiosk customer endpoint components as individual STL files.

#️⃣ Then, I sliced the exported parts in Bambu Studio, which provides an intuitive user interface for Bambu Lab printers to manage multi-color printing and PrusaSlicer-based features.

#️⃣ Since the vehicle platform does not require tensile strength, I decided to utilize the lightning infill pattern to reduce material use.

Step 15.a.2: Assembling the drive-through kiosk customer endpoint

After printing all of the kiosk customer endpoint parts successfully, I started to assemble them.

#️⃣ Since I purchased various well-known car replicas to represent different customer vehicles, I estimated the vehicle platform dimensions according to their measurements. Thus, the platform encompasses vehicles quite well :)

#️⃣ I inserted the four 20 mm steel balls (beads) into the stand base via the snug-fit holes.

#️⃣ Then, I connected the stand to the customer endpoint PCB case via four M3 screw-nut pairs.

#️⃣ Finally, I attached the customer endpoint PCB to its compact case via the special threaded bolts and nuts.

Step 15.b: Assigning a unique AprilTag to each food prep station

Since I aimed to design the food prep stations as a single unit, sharing the same tilted-down slider to transfer food items, I needed to utilize a visual fiducial system to differentiate each food prep station to enable the food delivery system gantry head to identify prep stations individually to collect the requested food items of the ordered menu.

Instead of creating custom visual identifiers from scratch, I decided to employ AprilTag markers, a well-established visual fiducial system popular in robotics research.

After inspecting the broad AprilTag marker catalogue, I assigned a marker to each food prep station as follows.

📌 Food Prep Station [1]

📌 Food Prep Station [2]

📌 Food Prep Station [3]

📌 Food Prep Station [4]

📌 Food Prep Station [5]

📌 Food Prep Station [6]

Step 15.c: Designing an assortment of color-coded figures representing each available food item

As mentioned in the previous steps, I enabled customers to select a deal (menu) preference from six different food categories. Each food category is assigned to a food prep station and includes three different food items, making eighteen food items in total for producing generic and user-specific (LLM-generated) menus/deals.

In a real-world drive-through restaurant establishment, restaurant cooks and workers would prepare the requested items and transfer them via the integrated tilted-down slider to ensure customers receive their orders via the automatic food delivery system. Nonetheless, since this is a proof-of-concept project, I wanted to manifest the food preparation and delivery process by utilizing scaled-down figures representing the available food items.

After mulling over what would be the most comprehensive design process to represent each food item with figures, I decided to create one unique figure (based on a simple shape) for each food category on Fusion 360, denoting the associated food prep station, and classify its three different food items by color during printing.

• 🍔 Burgers [Globe Figure]
• Hamburger ➡ 🔴 Red
• Cheeseburger ➡ 🟡 Yellow
• Veggie ➡ 🟢 Green
• 🌮 Mexican [Cylindrical Figure]
• Taco ➡ 🟡 Yellow
• Burrito ➡ 🔴 Red
• Tamales ➡ 🟢 Green
• 🍕 Italian [Circular Figure]
• Lasagna ➡ 🟡 Yellow
• Risotto ➡ 🟢 Green
• Pizza ➡ 🔴 Red
• 🦞 Seafood [Fish Figure]
• Shrimp Tempura ➡ 🟢 Green
• Salmon ➡ 🟡 Yellow
• Lobster ➡ 🔴 Red
• 🍟 Side Dishes [Cubic Figure]
• French Fries ➡ 🟡 Yellow
• Onion Rings ➡ 🟢 Green
• Chicken Nuggets ➡ 🔴 Red
• 🍰 Desserts [Triangular Figure]
• Brownie ➡ 🔴 Red
• Cheesecake ➡ 🟡 Yellow
• Carrot Cake ➡ 🟢 Green

Step 15.d: Designing the food prep stations, AprilTag signs, and the food carrier arm

#️⃣ First, I designed the restaurant workstation consisting of six different food prep stations as a single unit. It includes a tilted-down slider to pass the prepared food items to the food carrier arm (tray).

#️⃣ In this regard, the figures representing food items are placed on the top of the workstation, sorted by their food category, denoting the food prep station.

#️⃣ To make the AprilTag markers assigned to prep stations sturdy and swappable, I converted them to 3D models (signs) on Fusion 360 and designed a horizontal rack compatible with the workstation.

#️⃣ Finally, I designed the food carrier arm (tray) compatible with the Nema 17 (17HS3401) stepper motor shaft, making it easy to connect to the gantry head.

Step 15.d.1: Printing the custom parts of the food prep stations, the carrier arm, and the food figures

#️⃣ First, I exported the food prep stations (as a single unit), the horizontal AprilTag sign rack, and the food carrier arm as STL files to slice them on Bambu Studio.

#️⃣ Then, I exported the AprilTag signs and sliced them for multi-color printing by applying the built-in painting tool.

#️⃣ Since I specifically designed the food figures with multiple bodies (objects) on Fusion 360, I was able to make them behave as a single body on Bambu Studio and paint each object individually. I also sliced the special threaded bolts and nuts for the kiosk customer endpoint PCB case on this print plate since they share the same color with some of the figures.

Step 15.d.2: Assembling the food prep stations with AprilTag signs

#️⃣ As mentioned earlier, different colors represent different food items under the same food category, denoting food prep stations. I decided to print three figures for each food item.

#️⃣ I attached the horizontal AprilTag sign rack to the workstation (food prep stations as a single unit) via M3 screws and nuts. Then, I inserted the associated AprilTag signs into the rack for each food prep station from left to right.

#️⃣ Finally, I placed the food figures, representing the available food items, on the top of the workstation according to their food category, denoting the food prep station.

#️⃣ Thanks to the integrated tilted-down slider, shared by all food prep stations, the requested food items are transferred to the food carrier arm (tray). Please refer to the following steps to inspect the gantry head and the carrier arm assembly process.

Step 15.e: Designing the automatic food delivery system based on the H-Bot kinematic structure

As I was working on the automatic food delivery system of my drive-through restaurant establishment, I thoroughly researched different kinematic structures to move the ganty head with the food carrier arm precisely to deliver the requested food items to customers flawlessly.

In the final design process, I reduced my kinematic structure candidates to CoreXY (commonly used in 3D printers) and H-Bot. Even though CoreXY provides a more stable frame while moving the gantry, especially for devices requiring fast but precise millimetric movements, such as 3D printers, I decided to implement the H-Bot kinematic structure in my food delivery system due to these reasons:

• Since all food prep stations share the same tilted-down slider for passing the requested food items and food prep stations are classified via the assigned AprilTag signs, the H-Bot gantry mechanism is more than enough to provide the required accuracy to collect food items.
• Since the food delivery system gantry head should move relatively slowly and stably to avoid jerks while carrying food items, according to my frame design, CoreXY loses its torque distribution and fast acceleration advantages over H-Bot.
• H-Bot mechanisms are driven by two stepper motors and a single timing belt. Although CoreXY mechanisms are also driven by two stepper motors, they require two timing belts, leading to a more complicated design and more pulleys.

By H-Bot kinematic structure principles, the gantry head must interact with the single timing belt on opposite sides — front and back. While the start and end points of the single timing belt must be connected to the same side, the opposite side must pass through the timing belt.

The gantry head movements in relation to the stepper motors are as follows:

• The gantry head moves along the Cartesian X-axis while the stepper motors rotate at equal velocity in the same direction.
• The gantry head moves along the Cartesian Y-axis while the stepper motors rotate at equal velocity in opposite directions.
• The gantry head moves diagonally when one of the stepper motors rotates.

The H-Bot principles are beautifully demonstrated by Nguyen Duc Thang:

#️⃣ According to H-Bot kinematic structure principles, I designed the main frame of the automatic food delivery system in two parts (left and right) due to the build volume limits of the A1 Combo.

#️⃣ To provide a rigid connection between the main frame parts, I designed press-fit connectors supported not only by friction but also by M3 screws.

#️⃣ To balance the torque distribution while moving the gantry head, I designed X-axis and Y-axis rods as two (double) rows.

#️⃣ Due to the build volume limits, I designed the X-axis rods in two parts attached via snug-fit peg fasteners, supported not only by friction but also by M3 screws.

#️⃣ To establish the H-Bot structure, I designed unique axes connectors that secure the Y-axis and X-axis double rods.

#️⃣ Then, I designed the X-axis endstop, attached to the left axes connector, and the Y-axis endstop, attached to the left part of the main frame.

#️⃣ Finally, I designed lids for the stepper motors to reduce vibration while moving the gantry head.

#️⃣ As mentioned earlier, I decided to build the gantry head around a single Flex PCB. Since I had already designed the Flex PCB's pliable case, I began to work on designing the gantry head based on the bent (folded) state of the case.

#️⃣ I designed the gantry head in this unique shape since I needed to reduce material from the front of the gantry head to counterbalance the stepper motor controlling the food carrier arm (tray).

#️⃣ I made the gantry head compatible with GT2 aluminium timing belt clamps to attach the start and end points of the timing belt effortlessly.

#️⃣ While designing the gantry head, I ensured that the built-in camera of Nicla Vision and the food prep station AptilTag signs are aligned facing each other.

#️⃣ As I was designing my H-Bot-inspired food delivery system, I made sure the axes connector placements, the gantry head timing belt pass-through slot dimensions, and the stepper motor shaft heights were compatible with my GT2 pulley system components.

Step 15.e.1: Printing the custom parts of the H-Bot-inspired food delivery system, the gantry head, and the pliable Flex PCB case

#️⃣ First, I exported the main frame parts as STL files on Fusion 360. While slicing them on Bambu Studio, I accented the brand logos by applying the built-in painting tool and added tree supports to only the required areas by utilizing the built-in support blockers.

#️⃣ I exported the gantry head and the axes connectors as STL files and sliced them on Bambu Studio by modifying the tree support placement via the support blockers.

#️⃣ Then, I exported the main frame press-fit connectors, the axes rods, the X-axis rod snug-fit peg fasteners, the axes endstops, and the stepper motor lids as STL files. Since all of these components require high mechanical strength, I increased the wall loops (perimeters) to 6 while slicing.

#️⃣ Finally, I exported the unfolded version of the food delivery system Flex PCB case as an STL file and sliced it according to TPU 95A print settings.

Step 15.e.2: Assembling the delivery system gantry head and the Flex PCB case via M3 threaded inserts

#️⃣ First, I utilized my TS100 soldering iron to install M3 brass threaded inserts to the gantry head. Since threaded inserts use heat to bond by melting the plastic, they reinforce M3 screw connections more than any other method.

#️⃣ Since I printed the unfolded version of the delivery system Flex PCB case with TPU 95A, I was able to easily convert the case to its bent (folded) state to connect it to the gantry head via M3 screws through threaded inserts.

#️⃣ Finally, I added M3 brass hex standoffs to the gantry head through threaded inserts since I did not want to apply direct force to the bottom Flex PCB surface (without stiffeners) while affixing it to its case.

Step 15.e.3: Assembling the food delivery mechanism, placing the gantry head, and forming the H-Bot pulley system

#️⃣ First, I utilized M3 screws to assemble X-axis rods and their snug-fit peg fasteners. I also attached the X-axis endstop to the left axes connector via M3 screws.

#️⃣ I attached the left main frame part to the right main frame part via their press-fit connectors and strengthened the connection via M3 screws. Since I added simple guiding tenon joints to the main frame parts, I was able to easily attach the press-fit connectors.

#️⃣ Then, I attached the Y-axis endstop to the left part of the main frame via M3 screws.

#️⃣ After connecting the main frame, I started to assemble the H-Bot-inspired food delivery mechanism and its GT2 pulley system.

#️⃣ To build the H-Bot-inspired food delivery mechanism pulley system, I utilized these parts:

• 2 x GT2 20T Pulley (Drive)
• 4 x GT2 Toothless Passive Idler Pulley with Bearing (5mm Bore)
• 2 x GT2 16T Passive Idler Pulley with Bearing (3mm Bore)
• GT2 Timing Belt (5m)
• 2 x GT2 Aluminium Timing Belt Fixing Piece (Clamp)
• M3 Screws, Nuts, and Washers

#️⃣ First, I attached the stepper motor, controlling the food carrier arm, to the gantry head via M3 screws.

#️⃣ Then, I passed the X-axis rods through the provided gantry head slots and attached them to the axes connectors via M3 screws and nuts. Since I designed chamfered rods for both axes, near a semi-circle, I did not encounter any issues regarding jamming.

#️⃣ After fastening the food carrier arm (tray) to the stepper motor on the gantry head, I placed the Y-axis rods by passing them through the main frame and the axes connectors.

#️⃣ Then, I strengthened the Y-axis rod connections via M3 screws.

#️⃣ After concluding the frame assembly, I attached the stepper motor lids to the driver motors via M3 screws and fastened them to the provided slots on the main frame.

#️⃣ Then, I established the required GT2 pulley system allowing the two stepper motors to control the gantry head movement by driving a single timing belt according to the H-Bot kinetic structure principles.

#️⃣ Since I attached the passive (with bearing) GT2 pulleys via M3 screws, I was able to adjust the height differences between the drive and idler (passive) pulleys by adding M3 nuts. To reduce friction, I also utilized M3 washers.

#️⃣ Finally, I connected the start point of the single GT2 timing belt to the gantry head via a GT2 fixing piece (clamp) through the provided slot with an M3 screw.

#️⃣ After adjusting the timing belt length and tension, I connected the end point the same way.

Step 15.f: Designing, printing, and assembling the drive-through restaurant operation control panel (central)

As explained earlier, LattePanda Mu hosts the drive-through restaurant web application and runs large language models (LLMs) via Ollama to generate user-specific menus/deals, while Raspberry Pi 4 Model B manages the LR1302 LoRaWAN gateway.

To build a noteworthy drive-through restaurant operation control panel, I designed a simple sheet containing the mentioned development boards.

#️⃣ First, I designed the operation control panel on Fusion 360 based on the official LattePanda Mu evaluation board and Raspberry Pi 4 Model B CAD (step) files.

#️⃣ Then, I exported the control panel as an STL file. While slicing it on Bambu Studio, I accented the board areas, the panel borders, and the emblem by applying the built-in painting tool.

#️⃣ After printing the control panel, to avoid surface damage due to overheating, I installed M3 brass hex standoffs via M3 nuts and connected the development boards to the standoffs via M3 screws.

Step 16: Initial programming of RA-08H LoRaWAN node board

As mentioned earlier, I developed the automatic food delivery system and the kiosk customer endpoint features consecutively. Thus, the programming process was not linear, and the described code snippets below are only showcasing limited aspects of the presented code file. Please refer to the project GitHub page to review code files thoroughly.

📁 custom_variables.h

⭐ Define RGB565 color variables necessary to display the kiosk endpoint interface on the Fermion TFT display.

#define c_white_1     0xff7a
#define c_green_1     0x8c6b
#define c_green_2     0x746a
#define c_green_3     0x5e46
#define c_green_4     0x7de7
#define c_green_5     0xa70d
#define c_red_1       0xb123
#define c_red_2       0xda28
#define c_red_3       0xd2d2
#define c_blue_1      0x0377
#define c_blue_2      0x3454
#define c_blue_3      0x7677
#define c_blue_4      0x9f17
#define c_orange_1    0xf505
#define c_orange_2    0xf726
#define c_orange_3    0xed6f

#define c_edge_b      0x2d99
#define c_edge_g      0x9de0
#define c_edge_y      0xfda0
#define c_edge_r      0xfb05

📁 ai_driven_customer_service_kiosk_lora_end_point.ino

⭐ Include the required libraries.

#include <SPI.h>
#include <SD.h>
#include "DFRobot_GDL.h"
#include "DFRobot_Picdecoder_SD.h"
#include <Adafruit_VC0706.h>
#include <Keypad.h>

⭐ Import the custom RGB565 color variables.

#include "custom_variables.h"

⭐ Define the required Fermion TFT display SPI pin configurations and enable the select pin for the built-in MicroSD card reader.

⭐ Then, define the Fermion display object and the integrated JPG image decoder from the microSD card.

#define SD_CS_PIN 15

// Define the Fermion TFT LCD display pin configurations.
#define TFT_DC  21
#define TFT_CS  17
#define TFT_RST 20

// Define the Fermion TFT LCD display object and integrated JPG decoder.
DFRobot_Picdecoder_SD decoder;
DFRobot_ST7789_240x320_HW_SPI screen(/*dc=*/TFT_DC,/*cs=*/TFT_CS,/*rst=*/TFT_RST);
#define SCREEN_WIDTH   240
#define SCREEN_HEIGHT  320

⭐ Define the VC0706 (Crowtail serial) camera object and assign the second hardware serial port of RP2040 for data transfer since the first hardware serial port is reserved for the onboard RA-08H LoRaWAN module.

Adafruit_VC0706 serial_cam = Adafruit_VC0706(&Serial2);

⭐ Declare two different custom 4x4 keypad maps (layouts) — numbers and letters — compatible with 16 digits of hexadecimal numbers. I also added special characters to both layouts to enable user interactions to operate the kiosk endpoint interface.

const byte ROWS = 4; // row_number
const byte COLS = 4; // column_number

// Define the custom keypad symbols with different configurations.
char num_Keys[ROWS][COLS] = {
  {'1','2','3','>'}, // [1, 2, 3, NEXT]
  {'4','5','6','<'}, // [4, 5, 6, PREVIOUS]
  {'7','8','9','='}, // [7, 8, 9, SELECT]
  {'+','0','-','!'}  // [ACTIVATE, 0, DELETE, EXIT]
};
char let_Keys[ROWS][COLS] = {
  {'a','b','c','>'}, // [A, B, C, NEXT]
  {'d','e','f','<'}, // [D, E, F, PREVIOUS]
  {'x','x','x','='}, // [X, X, X, SELECT]
  {'+','x','-','!'}  // [ACTIVATE, X, DELETE, EXIT]
};

// Define the custom keypad row and column pins.
byte rowPins[ROWS] = {6, 7, 10, 11};
byte colPins[COLS] = {22, 23, 24, 25};

// Initialize the custom keypad maps with the given settings — numbers and letters.
Keypad num_keypad = Keypad(makeKeymap(num_Keys), rowPins, colPins, ROWS, COLS);
Keypad let_keypad = Keypad(makeKeymap(let_Keys), rowPins, colPins, ROWS, COLS);

#️⃣ As explained in the previous steps, I needed to register JoinEUI (formerly called AppEUI), DevEUI, and AppKey variables to create an end device (node) on my TTN console.

⭐ In this regard, declare the registered variables by creating a struct to program the RA-08H LoRaWAN module to connect to The Things Network as an end device through the LR1302 LoRaWAN gateway.

struct ttn_settings{
  String DEVEUI = "TTN_DEVEUI"; // e.g., 60B3C58ED016FF89
  String APPEUI = "TTN_APPEUI"; // e.g., 1111111111111111
  String APPKEY = "TTN_APPKEY"; // e.g., 39C4BC565188ABE36AFC123754C42FB0
  boolean ra_08h_update_init = true;
  boolean ra_08h_update_ongoing = false;
  int _delay = 500;
  char con_status[8] = {'w', 'w', 'w', 'w', 'w', 'w', 'w', 'w'};
};

⭐ Initiate the first hardware serial port of RP2040 with its default RX and TX pins to communicate with the onboard RA-08H module.

⭐ Initiate the second hardware serial port of RP2040 with its default RX and TX pins to communicate with the Crowtail serial (UART) camera.

⭐ Configure the serial camera picture resolution to 320 x 240 due to memory limitations.

Serial1.setRX(1); Serial1.setTX(0);
  Serial1.begin(9600);
  
  ...
 
  Serial2.setRX(9); Serial2.setTX(8);
  
  ...
  
  serial_cam.setImageSize(VC0706_320x240);

⭐ In the ra_08h_send_to_lora_gateway function:

⭐ Transmit the passed data packet to The Things Network through the LR1302 LoRaWAN gateway by utilizing the assigned AT command.

⭐ Notify the user of the state of the LoRa data packet transfer.

bool ra_08h_send_to_lora_gateway(String data_packet){
  String data_res = "";
  // Send the passed data packet to the connected LR1302 LoRaWAN Gateway Module.
  Serial1.print("AT+DTRX=1,2,3," + data_packet + "\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; data_res = ra_08h_update_response(); Serial.println(data_res);
  // Notify the user of the LoRa data packet transfer success.
  if(data_res.indexOf("ERR") >= 0){ adjustColor(255,0,0); return false; }
  else{ adjustColor(0,255,0); return true; }
}

⭐ In the configure_ra_08h_lora_settings function:

⭐ Configure the RA-08H LoRaWAN node module by utilizing the assigned AT commands (serial communication) according to the provided TTN-registered end device information — DEVEUI, JOINEUI (formerly called APPEUI), and APPKEY.

⭐ After obtaining the AT command responses from the RA-08H module, change the status of the associated commands accordingly.

void configure_ra_08h_lora_settings(String DEVEUI, String APPEUI, String APPKEY){
  String update_res = "";
  Serial.println("RA-08H LoRa Setting Configuration Started...\n");
  adjustColor(255,0,255);
  /*
    Update the RA-08H settings via AT commands according to the generated TTN server application — DEVEUI, JOINEUI (formerly called APPEUI), and APPKEY.
    After obtaining the response from the RA-08H LoRa module, change the update status of the associated command accordingly. 
  */
  // Set the node access method to OTAA
  Serial1.print("AT+CJOINMODE=0\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[0] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Set the node group frequency mask (Set channels 0-7)
  Serial1.print("AT+CFREQBANDMASK=0002\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[1] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Set the node type
  Serial1.print("AT+CCLASS=2\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[2] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Assign the associated TTN application end device DEVEUI
  Serial1.print("AT+CDEVEUI=" + DEVEUI + "\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[3] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Assign the associated TTN application end device JOINEUI (formerly called APPEUI)
  Serial1.print("AT+CAPPEUI=" + APPEUI + "\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[4] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Assign the associated TTN application end device APPKEY
  Serial1.print("AT+CAPPKEY=" + APPKEY + "\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[5] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Set the node uplink and downlink frequency — 1 for same frequency, 2 for different frequency 
  Serial1.print("AT+CULDLMODE=2\n"); delay(ttn_settings._delay);  
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[6] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Set the node to join the network
  Serial1.print("AT+CJOIN=1,1,10,3\n"); delay(ttn_settings._delay); 
  ttn_settings.ra_08h_update_ongoing = true; update_res = ra_08h_update_response(); Serial.println(update_res);
  ttn_settings.con_status[7] = (update_res.indexOf("ERR") >= 0 || update_res.indexOf("FAIL") >= 0) ? 'f' : 's';
  // Update completed.
  Serial.println("RA-08H LoRa Setting Configuration Ended...\n");
  adjustColor(0,255,0);
}

⭐ In the ra_08h_update_response function:

⭐ Wait until the RA-08H module responds to the sent AT commands via serial communication.

⭐ Halt the loop once the RA-08H module responds or does not send a data packet during the given timeframe.

⭐ Finally, return the retrieved response.

String ra_08h_update_response(){
  String ra_08h_response = "No response!";
  int port_wait = 0;
  // Wait until RA-08H transfers a response after sending AT commands via serial communication.
  while(ttn_settings.ra_08h_update_ongoing){
    port_wait++;
    if(Serial1.available()){
      ra_08h_response = Serial1.readString();
    }
    // Halt the loop if the RA-08H returns a data packet or does not respond in the given timeframe.
    if(ra_08h_response != "" || port_wait > 30000){
      ttn_settings.ra_08h_update_ongoing = false;
    }
  }
  // Then, return the retrieved response.
  delay(500);
  return ra_08h_response;
}

⭐ In the capture_and_save_picture function:

⭐ Attempt to capture a snapshot (image frame) from the video stream produced by the Crowtail serial camera (VC0706).

⭐ Once capturing the frame successfully, obtain the length of the image buffer.

⭐ To save the captured image buffer to the microSD card (integrated into the Fermion TFT display), create a JPG file name with the passed account authentication key and the assigned sample number.

⭐ Then, according to the estimated image buffer length (total byte number), write the retrieved buffer to the created image file 32 bytes at a time.

⭐ Record the success of the image-capturing process in the image sample collection status array based on the given sample number.

⭐ Reset the Crowtail serial camera (VC0706) after taking a snapshot to clear the integrated image buffer. Otherwise, the serial camera cannot yield accurate image buffers.

void capture_and_save_picture(String account_key, int sample_num){
  adjustColor(255,255,0);
  if(serial_cam.takePicture()){
    // If the Crowtail serial camera (VC0706) successfully captured a frame (image):
    Serial.println("Crowtail => Image captured successfully!");
    // After capturing the frame, obtain the length of the image buffer.
    uint32_t buf_len = serial_cam.frameLength();
    // Then, save the image buffer to the SD card with the passed account authentication key and the image sample number — in the JPG format.
    String filename = "vehicle_image_samples/" + account_key + "__" + String(sample_num+1) + ".jpg";
    File new_img_file = SD.open(filename.c_str(), "w");
    // According to the obtained image buffer length (total byte number), write the retrieved buffer to the new image file 32 bytes at a time.
    byte byte_n = 0;
    while(buf_len > 0){
      uint8_t *img_buffer;
      uint8_t currentBytes = min((uint32_t)32, buf_len);
      img_buffer = serial_cam.readPicture(currentBytes);
      new_img_file.write(img_buffer, currentBytes);
      if(++byte_n >= 64){ Serial.print('.'); byte_n = 0; }
      buf_len -= currentBytes;
    } new_img_file.close();
    // Capture success.
    img_save_status[sample_num] = 's';
    Serial.println("Crowtail => Image saved to the SD card successfully!");
    adjustColor(0,255,0);    
  }else{
    // Capture failure.
    img_save_status[sample_num] = 'f';
    Serial.println("Crowtail => Cannot capture image!");
    adjustColor(255,0,0);
  }
  // Reset the Crowtail serial camera (VC0706) after taking a snaphot to clear the image frame.
  serial_cam.reset();
}

⭐ I programmed the show_screen function to showcase all of the kiosk endpoint interface menus, state changes, status changes, and keypad inputs on the Fermion TFT display.

void show_screen(String _type, int _opt, volatile boolean _clear){
  // According to the given parameters, show the requested screen type on the Fermion TFT LCD display.
  int pic_w = 60, pic_h = 60, pic_f = 8;
  int h_y_s = 25;
  int text_s_s = 6, text_m_s = 12, text_b_s = 18, text_bb_s = 24;
  if(_type == "init"){
    adjustColor(0,0,0);
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRoundRect(0, h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    decoder.drawPicture("assets/lora_icon.bmp", pic_f/2, h_y_s+pic_f/2, pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    screen.setTextColor(c_orange_1);
    screen.setTextSize(3);
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_f);
    screen.print("LoRa");
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_h+pic_f-22);
    screen.print("Kiosk");
    screen.fillRoundRect(SCREEN_WIDTH-(pic_w+pic_f), h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    String opt_img = "assets/menu_logo_" + String(_opt) + ".bmp";
    decoder.drawPicture(opt_img.c_str(), (SCREEN_WIDTH-(pic_w+pic_f))+pic_f/2, h_y_s+pic_f/2, (SCREEN_WIDTH-(pic_w+pic_f))+pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    int y_end = h_y_s+pic_h+pic_f, x_str = 32, init_sp = 45, l_sp = 40;
    int c_r = 10, c_x_s = x_str - c_r - 10, c_r_sp = 4, c_l_sp = 5;
    screen.setTextSize(2);
    int highlight_c = (_opt == 1) ? c_blue_4 : c_blue_3;
    int border_c = c_orange_1;
    y_end += init_sp; screen.fillCircle(c_x_s, y_end+c_l_sp, c_r, border_c); screen.fillCircle(c_x_s, y_end+c_l_sp, c_r-c_r_sp, highlight_c);
    screen.setCursor(x_str, y_end);
    screen.setTextColor(highlight_c);
    screen.print("Order Menu");
    highlight_c = (_opt == 2) ? c_blue_4 : c_blue_3;
    y_end += l_sp; screen.fillCircle(c_x_s, y_end+c_l_sp, c_r, border_c); screen.fillCircle(c_x_s, y_end+c_l_sp, c_r-c_r_sp, highlight_c);
    screen.setCursor(x_str, y_end);
    screen.setTextColor(highlight_c);
    screen.print("Validate Vehicle");    
    highlight_c = (_opt == 3) ? c_blue_4 : c_blue_3;
    y_end += l_sp; screen.fillCircle(c_x_s, y_end+c_l_sp, c_r, border_c); screen.fillCircle(c_x_s, y_end+c_l_sp, c_r-c_r_sp, highlight_c);
    screen.setCursor(x_str, y_end);
    screen.setTextColor(highlight_c);
    screen.print("Add New Vehicle"); 
    highlight_c = (_opt == 4) ? c_blue_4 : c_blue_3;
    y_end += l_sp; screen.fillCircle(c_x_s, y_end+c_l_sp, c_r, border_c); screen.fillCircle(c_x_s, y_end+c_l_sp, c_r-c_r_sp, highlight_c);
    screen.setCursor(x_str, y_end);
    screen.setTextColor(highlight_c);
    screen.print("Reset Connection"); 
  }
  
  else if(_type == "order_menu"){  
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRoundRect(0, h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    String opt_img = "assets/menu_logo_" + String(_opt) + ".bmp";
    decoder.drawPicture(opt_img.c_str(), pic_f/2, h_y_s+pic_f/2, pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    screen.setTextColor(c_orange_1);
    screen.setTextSize(3);
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_f);
    screen.print("Order");
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_h+pic_f-22);
    screen.print("Menu");
    int y_end = h_y_s+pic_h+pic_f, init_sp = 55;
    int tri_x_sp = 5, tri_w = 20, tri_h = 30, tri_sc = 4;
    int x_str = tri_x_sp + tri_w + 10;
    y_end += init_sp;
    screen.setTextColor(c_white_1);
    screen.setTextSize(2);
    screen.fillTriangle(tri_x_sp, y_end-(tri_h/2), tri_x_sp+tri_w, y_end, tri_x_sp, y_end+(tri_h/2), c_green_3);
    screen.fillTriangle(tri_x_sp+(tri_sc/2), y_end-(tri_h/2)+tri_sc, tri_x_sp+tri_w-tri_sc, y_end, tri_x_sp+(tri_sc/2), y_end+(tri_h/2)-tri_sc, c_white_1);    
    screen.setCursor(x_str, y_end-(text_m_s/2));
    screen.print("Enter Menu Code");
    int l_str = (SCREEN_WIDTH - (text_bb_s * order_menu_command.length())) / 2;
    y_end += (tri_h/2) + (text_bb_s*1.5);
    screen.fillRect(0, y_end-5, SCREEN_WIDTH, text_bb_s+10, c_green_1);
    int _hightlight_c = (order_menu_command.length() == 4) ? c_green_3 : c_white_1;
    screen.setTextColor(_hightlight_c);
    screen.setTextSize(4);
    screen.setCursor(l_str, y_end);
    screen.print(order_menu_command);
    int f_text_sp = 5;
    y_end = SCREEN_HEIGHT - f_text_sp - (text_s_s*2);
    screen.fillRect(0, y_end-5, text_s_s*8, (text_s_s*2)+f_text_sp, c_green_1);
    screen.setTextSize(1);
    screen.setTextColor(c_orange_2);
    screen.setCursor(f_text_sp, y_end);
    String cur_keypad = (current_keypad_map == 1) ? "K: 123" : "K: ABC";
    screen.print(cur_keypad);
    String auth_status = (vehicle_validation_status) ? "VEHICLE: AUTHORIZED" : "VEHICLE: NO_AUTH";
    int auth_color = (vehicle_validation_status) ? c_green_3 : c_red_1;
    int f_text_right_x_str = SCREEN_WIDTH - (auth_status.length()*text_s_s) - f_text_sp;
    screen.setTextColor(auth_color);
    screen.setCursor(f_text_right_x_str, y_end);
    screen.print(auth_status);
  }

  else if(_type == "order_menu_success"){
    int b_pic_w = pic_w*2, b_pic_h = pic_h*2, b_pic_f = pic_f*2, b_pic_y_sp = 15;
    int y_sp = 15, y_l_sp = 30;
    int y_end = y_sp;
    if(_clear) screen.fillScreen(c_green_1);
    screen.setCursor(0, y_end);
    screen.setTextColor(c_green_3);
    screen.setTextSize(3);
    screen.print("Menu: " + order_menu_command);
    y_end += text_bb_s + y_l_sp;
    screen.setCursor(0, y_end);
    screen.setTextColor(c_green_5);
    screen.setTextSize(1);
    screen.println("The LoRa code of the requested menu has been successfully transferred \nvia the established LoRaWAN network.\n");
    screen.print("Please inspect your web dashboard to \ntrack the order status and receive the \nrequested menu / deal. The payment \nprocess is handled via the provided \naccount settings.");
    y_end = SCREEN_HEIGHT - b_pic_y_sp - b_pic_h - b_pic_f;
    int b_pic_x_str = (SCREEN_WIDTH/2) - ((b_pic_w+b_pic_f)/2);
    screen.fillRoundRect(b_pic_x_str, y_end, b_pic_w+b_pic_f, b_pic_w+b_pic_f, b_pic_f/2, c_green_4);
    decoder.drawPicture("assets/order_success.bmp", b_pic_x_str+(b_pic_f/2), y_end+(b_pic_f/2), b_pic_x_str+(b_pic_f/2)+b_pic_w, y_end+(b_pic_f/2)+b_pic_h, screenDrawPixel);    
  }

  else if(_type == "order_menu_failure"){
    int b_pic_w = pic_w*2, b_pic_h = pic_h*2, b_pic_f = pic_f*2, b_pic_y_sp = 15;
    int y_sp = 15, y_l_sp = 30;
    int y_end = y_sp;
    if(_clear) screen.fillScreen(c_green_1);
    screen.setCursor(0, y_end);
    screen.setTextColor(c_red_2);
    screen.setTextSize(3);
    screen.print("Menu: " + order_menu_command);
    y_end += text_bb_s + y_l_sp;
    screen.setCursor(0, y_end);
    screen.setTextColor(c_red_1);
    screen.setTextSize(1);
    screen.println("Cannot send the given code via the \nLoRaWAN network.\n\nPlease reset the module connection\nsettings via the user interface.");
    y_end = SCREEN_HEIGHT - b_pic_y_sp - b_pic_h - b_pic_f;
    int b_pic_x_str = (SCREEN_WIDTH/2) - ((b_pic_w+b_pic_f)/2);
    screen.fillRoundRect(b_pic_x_str, y_end, b_pic_w+b_pic_f, b_pic_w+b_pic_f, b_pic_f/2, c_red_3);
    decoder.drawPicture("assets/order_failure.bmp", b_pic_x_str+(b_pic_f/2), y_end+(b_pic_f/2), b_pic_x_str+(b_pic_f/2)+b_pic_w, y_end+(b_pic_f/2)+b_pic_h, screenDrawPixel);     
  }

  else if(_type == "valid_vehicle"){
    int _pic_w = 120, _pic_h = 120, _pic_f = 15, _pic_y_sp = 15;
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRoundRect(0, h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    String opt_img = "assets/menu_logo_" + String(_opt) + ".bmp";
    decoder.drawPicture(opt_img.c_str(), pic_f/2, h_y_s+pic_f/2, pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    screen.setTextColor(c_orange_1);
    screen.setTextSize(3);
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_f);
    screen.print("Validate");
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_h+pic_f-22);
    screen.print("Vehicle");
    int y_end = h_y_s+pic_h+pic_f, init_sp = 35;
    y_end += init_sp;
    String highlight_mes, highlight_pic; int highlight_c;
    if(vehicle_validation_status){
      highlight_c = c_edge_g;
      highlight_mes = "Already validated your registered\nvehicle; thanks for your patronage :)\nClick again to discard validation!";
      highlight_pic = "assets/run_inference_success.bmp";      
    }else{
      highlight_c = c_edge_b;
      highlight_mes = "To order menus and campaigns\nspecifically generated for you, please\nauthorize your registered vehicle!";
      highlight_pic = "assets/run_inference_idle.bmp";
    }
    screen.setTextColor(highlight_c);
    screen.setTextSize(1);
    screen.setCursor(0, y_end);
    screen.print(highlight_mes);
    y_end = SCREEN_HEIGHT - _pic_h - _pic_f/2 - _pic_y_sp;
    int _pic_x_str = (SCREEN_WIDTH - (_pic_w + _pic_f)) / 2;
    screen.fillRoundRect(_pic_x_str, y_end, _pic_w+_pic_f, _pic_h+_pic_f, _pic_f/2, highlight_c);
    decoder.drawPicture(highlight_pic.c_str(), _pic_x_str+(_pic_f/2), y_end+(_pic_f/2), _pic_x_str+(_pic_f/2)+_pic_w, y_end+(_pic_f/2)+_pic_h, screenDrawPixel);
  }
  
  else if(_type == "valid_vehicle_run"){
    int _pic_w = 80, _pic_h = 80, _pic_f = 10, _pic_x_sp = 5, _pic_y_sp = 35;
    int y_end = _pic_y_sp;
    int _pic_x_str = _pic_x_sp;
    int hor_line_w = SCREEN_WIDTH - _pic_w  - _pic_f, hor_line_h = _pic_f/2;
    int hor_line_x_str = _pic_x_str + (_pic_f/2) + (_pic_w/2), hor_line_y_str = y_end + (_pic_f/2) + (_pic_h/2) - (hor_line_h/2);
    int highlight_c = (inference_status == 'i') ? c_edge_y : ((inference_status == 'n') ? c_edge_r : c_edge_g);
    String highlight_mes = (inference_status == 'i') ? "Running..." : ((inference_status == 'n') ? "Not registered!" : "Vehicle Code: " + detected_class);
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRect(hor_line_x_str, hor_line_y_str, hor_line_w, hor_line_h, highlight_c);
    screen.fillRoundRect(_pic_x_str, y_end, _pic_w+_pic_f, _pic_h+_pic_f, _pic_f/2, highlight_c);
    decoder.drawPicture("assets/edge_impulse_icon.bmp", _pic_x_str+(_pic_f/2), y_end+(_pic_f/2), _pic_x_str+(_pic_f/2)+_pic_w, y_end+(_pic_f/2)+_pic_h, screenDrawPixel);
    _pic_x_str = SCREEN_WIDTH - _pic_f - _pic_w - _pic_x_sp;
    screen.fillRoundRect(_pic_x_str, y_end, _pic_w+_pic_f, _pic_h+_pic_f, _pic_f/2, highlight_c);
    decoder.drawPicture("assets/run_inference_cam.bmp", _pic_x_str+(_pic_f/2), y_end+(_pic_f/2), _pic_x_str+(_pic_f/2)+_pic_w, y_end+(_pic_f/2)+_pic_h, screenDrawPixel);
    int ver_line_w = _pic_f/2, ver_line_h = 1.5*_pic_h;
    int ver_line_x_str = (SCREEN_WIDTH - ver_line_w) / 2, ver_line_y_str = hor_line_y_str;
    y_end = ver_line_y_str + ver_line_h - (_pic_h/2) - (_pic_f/2);
    _pic_x_str = (SCREEN_WIDTH - _pic_w - _pic_f) / 2;
    if(inference_status != 'i'){
      String highlight_pic = (inference_status == 'd') ? "assets/run_inference_detected.bmp" : "assets/run_inference_not_detected.bmp";
      screen.fillRect(ver_line_x_str, ver_line_y_str, ver_line_w, ver_line_h, highlight_c);
      screen.fillRoundRect(_pic_x_str, y_end, _pic_w+_pic_f, _pic_h+_pic_f, _pic_f/2, highlight_c);
      decoder.drawPicture(highlight_pic.c_str(), _pic_x_str+(_pic_f/2), y_end+(_pic_f/2), _pic_x_str+(_pic_f/2)+_pic_w, y_end+(_pic_f/2)+_pic_h, screenDrawPixel);
    }
    int l_sp = 25, l_x_str = (SCREEN_WIDTH - (highlight_mes.length()*text_m_s)) / 2;
    y_end += _pic_h + _pic_f + l_sp;
    screen.setTextColor(highlight_c);
    screen.setTextSize(2);
    screen.setCursor(l_x_str, y_end);
    screen.fillRect(0, y_end-5, SCREEN_WIDTH, text_m_s+10, c_green_1);
    screen.print(highlight_mes);
    y_end += text_m_s*2;
    if(inference_status == 'd' && vehicle_validation_status){
      String code_lora_success = "LoRa => Transferred!";
      l_x_str = (SCREEN_WIDTH - (code_lora_success.length()*text_m_s)) / 2;
      screen.setTextColor(c_edge_b);
      screen.setCursor(l_x_str, y_end);
      screen.print(code_lora_success);      
    }
    if(inference_status == 'd' && !vehicle_validation_status){
      String code_lora_success = "LoRa => Failed!";
      l_x_str = (SCREEN_WIDTH - (code_lora_success.length()*text_m_s)) / 2;
      screen.setTextColor(c_edge_r);
      screen.setCursor(l_x_str, y_end);
      screen.print(code_lora_success); 
    }
  }  

  else if(_type == "add_vehicle"){
    // Enable showing the modified account authentication key if the user did not validate a registered vehicle.
    String _account_auth_key = (!vehicle_validation_status) ? account_auth_key : "AUTH"; 
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRoundRect(0, h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    String opt_img = "assets/menu_logo_" + String(_opt) + ".bmp";
    decoder.drawPicture(opt_img.c_str(), pic_f/2, h_y_s+pic_f/2, pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    screen.setTextColor(c_orange_1);
    screen.setTextSize(3);
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_f);
    screen.print("Add");
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_h+pic_f-22);
    screen.print("Vehicle");
    int y_end = h_y_s+pic_h+pic_f, init_sp = 55;
    int tri_x_sp = 5, tri_w = 20, tri_h = 30, tri_sc = 4;
    int x_str = tri_x_sp + tri_w + 10;
    y_end += init_sp;
    screen.setTextColor(c_white_1);
    screen.setTextSize(2);
    screen.fillTriangle(tri_x_sp, y_end-(tri_h/2), tri_x_sp+tri_w, y_end, tri_x_sp, y_end+(tri_h/2), c_green_3);
    screen.fillTriangle(tri_x_sp+(tri_sc/2), y_end-(tri_h/2)+tri_sc, tri_x_sp+tri_w-tri_sc, y_end, tri_x_sp+(tri_sc/2), y_end+(tri_h/2)-tri_sc, c_white_1);    
    screen.setCursor(x_str, y_end-(text_m_s/2));
    screen.print("Enter Account Key");
    int l_str = (SCREEN_WIDTH - (text_bb_s * _account_auth_key.length())) / 2;
    y_end += (tri_h/2) + (text_bb_s*1.5);
    screen.fillRect(0, y_end-5, SCREEN_WIDTH, text_bb_s+10, c_green_1);
    int _hightlight_c = (_account_auth_key.length() == 4) ? c_green_3 : c_white_1;
    screen.setTextColor(_hightlight_c);
    screen.setTextSize(4);
    screen.setCursor(l_str, y_end);
    screen.print(_account_auth_key);
    int f_text_sp = 5;
    y_end = SCREEN_HEIGHT - f_text_sp - (text_s_s*2);
    screen.fillRect(0, y_end-5, text_s_s*8, (text_s_s*2)+f_text_sp, c_green_1);
    screen.setTextSize(1);
    screen.setTextColor(c_orange_2);
    screen.setCursor(f_text_sp, y_end);
    String cur_keypad = (current_keypad_map == 1) ? "K: 123" : "K: ABC";
    screen.print(cur_keypad);
    String auth_status = (vehicle_validation_status) ? "VEHICLE: AUTHORIZED" : "VEHICLE: NO_AUTH";
    int auth_color = (vehicle_validation_status) ? c_green_3 : c_red_1;
    int f_text_right_x_str = SCREEN_WIDTH - (auth_status.length()*text_s_s) - f_text_sp;
    screen.setTextColor(auth_color);
    screen.setCursor(f_text_right_x_str, y_end);
    screen.print(auth_status);
  }

  else if(_type == "add_vehicle_img_collect"){
    int _pic_w = 80, _pic_h = 80, _pic_f = 10, _pic_y_sp = 30;
    int y_sp = 15, y_l_sp = 30;
    int y_end = y_sp;
    if(_clear) screen.fillScreen(c_green_1);
    screen.setCursor(0, y_end);
    screen.setTextColor(c_blue_1);
    screen.setTextSize(3);
    screen.print("Account: " + account_auth_key);
    y_end += text_b_s + _pic_y_sp;
    int _p_pic_x_str = (SCREEN_WIDTH - _pic_w) / 2;
    screen.fillRoundRect(_p_pic_x_str-(_pic_f/2), y_end-(_pic_f/2), _pic_w+_pic_f, _pic_h+_pic_f, _pic_f, c_blue_2);
    decoder.drawPicture("assets/camera_icon.bmp", _p_pic_x_str, y_end, _p_pic_x_str+_pic_w, y_end+_pic_h, screenDrawPixel);
    y_end += _pic_h + (_pic_f/2) + _pic_y_sp - 10;
    int x_str = 15, l_sp = 12;
    int img_status_c = (img_save_status[0] == 'w') ? c_orange_2 : ((img_save_status[0] == 'f') ? c_red_1 : c_green_3);
    String img_file_status = (img_save_status[0] == 'w') ? "Capturing sample..." : ((img_save_status[0] == 'f') ? "Error => Cannot capture image!" : "Successful => "+account_auth_key+"_img_sample_1.jpg");
    screen.fillRect(0, y_end-text_s_s, SCREEN_WIDTH, text_s_s*3, c_green_1);
    screen.setTextSize(1);
    screen.setCursor(x_str, y_end);
    screen.setTextColor(img_status_c);
    screen.print(img_file_status);
    for(int i = 1; i < IMG_SAMPLE_NUM; i++){
      y_end += (text_s_s*3) + l_sp;
      img_status_c = (img_save_status[i] == 'w') ? c_orange_2 : ((img_save_status[i] == 'f') ? c_red_1 : c_green_3);
      img_file_status = (img_save_status[i] == 'w') ? "Capturing sample..." : ((img_save_status[i] == 'f') ? "Error => Cannot capture image!" : "Successful => "+account_auth_key+"_img_sample_"+String(i+1)+".jpg");
      screen.fillRect(0, y_end-text_s_s, SCREEN_WIDTH, text_s_s*3, c_green_1);    
      screen.setCursor(x_str, y_end);
      screen.setTextColor(img_status_c);
      screen.print(img_file_status);  
    }
  }

  else if(_type == "reset_con"){  
    if(_clear) screen.fillScreen(c_green_1);
    screen.fillRoundRect(0, h_y_s, pic_w+pic_f, pic_h+pic_f, pic_f/2, c_orange_3);
    String opt_img = "assets/menu_logo_" + String(_opt) + ".bmp";
    decoder.drawPicture(opt_img.c_str(), pic_f/2, h_y_s+pic_f/2, pic_w+pic_f/2, h_y_s+pic_h+pic_f/2, screenDrawPixel);
    screen.setTextColor(c_orange_1);
    screen.setTextSize(3);
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_f);
    screen.print("Reset");
    screen.setCursor(pic_w+pic_f+11, h_y_s+pic_h+pic_f-22);
    screen.print("LoRa Con");
    int y_end = h_y_s+pic_h+pic_f, init_sp = 35, x_str = 10;
    int sq_w = 15, sq_x_sp = 10, sq_y_sp = 8, sq_r = 3, sq_x_str = SCREEN_WIDTH-sq_w-sq_x_sp;
    y_end += init_sp + sq_y_sp;
    int sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setTextSize(1);
    screen.setTextColor(c_blue_4);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CJOINMODE");
    int status_c = (ttn_settings.con_status[0] == 'w') ? c_orange_2 : ((ttn_settings.con_status[0] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CFREQBANDMASK");
    status_c = (ttn_settings.con_status[1] == 'w') ? c_orange_2 : ((ttn_settings.con_status[1] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CCLASS");
    status_c = (ttn_settings.con_status[2] == 'w') ? c_orange_2 : ((ttn_settings.con_status[2] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CDEVEUI");
    status_c = (ttn_settings.con_status[3] == 'w') ? c_orange_2 : ((ttn_settings.con_status[3] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);  
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CAPPEUI");
    status_c = (ttn_settings.con_status[4] == 'w') ? c_orange_2 : ((ttn_settings.con_status[4] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);     
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CAPPKEY");
    status_c = (ttn_settings.con_status[5] == 'w') ? c_orange_2 : ((ttn_settings.con_status[5] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);    
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CULDLMODE");
    status_c = (ttn_settings.con_status[6] == 'w') ? c_orange_2 : ((ttn_settings.con_status[6] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);    
    y_end += text_s_s + (sq_w/2) + sq_y_sp;
    sq_y_str = y_end-((sq_w-text_s_s) / 2);
    screen.setCursor(x_str, y_end);
    screen.print("AT+CJOIN");
    status_c = (ttn_settings.con_status[7] == 'w') ? c_orange_2 : ((ttn_settings.con_status[7] == 'f') ? c_red_1 : c_green_3);
    screen.fillRoundRect(sq_x_str, sq_y_str, sq_w, sq_w, sq_r, status_c);
  }  
}

⭐ Initiate the kiosk customer endpoint interface and show the home screen.

if(scr_init){
    show_screen("init", selected_interface_opt, scr_init);
  } scr_init = false;

⭐ Obtain the pressed keycap value assigned by the number keypad layout (map).

⭐ Update the highlighted kiosk endpoint interface option once the '>' (NEXT) or '<' (PREVIOUS) keycap is pressed.

⭐ Activate the highlighted interface option when the '=' (SELECT) keycap is pressed.

char activeKey = num_keypad.getKey();
  if(activeKey != NO_KEY){
    // Update the highlighted interface option if the '>' (NEXT) or '<' (PREVIOUS) keycaps are pressed.
    if(activeKey == '>'){
      selected_interface_opt--;
      if(selected_interface_opt < 0) selected_interface_opt = 4;
      show_screen("init", selected_interface_opt, false);
    }     
    if(activeKey == '<'){
      selected_interface_opt++;
      if(selected_interface_opt > 4) selected_interface_opt = 0;
      show_screen("init", selected_interface_opt, false);
    }
    // Select the highlighted interface option if the '=' (SELECT) keycap is pressed.
    if(activeKey == '=' && selected_interface_opt > 0){
      active_int_opt[selected_interface_opt-1] = true;
    }  
  }

⭐ If the order_menu endpoint interface option is activated:

⭐ Obtain the pressed keycap value according to the selected keypad map (layout) — numbers or letters.

⭐ Select the numbers keypad map once the '>' (NEXT) keycap is pressed.

⭐ Select the letters keypad map once the '<' (PREVIOUS) keycap is pressed.

⭐ Notify the user of the current keypad map selection on the screen.

⭐ When pressed, add the appropriate keycap values (number or letter) to the order menu string, representing the 4-digit order tag assigned by the restaurant web application.

⭐ Discard the latest character from the order menu string if the '-' (DELETE) keycap is pressed.

⭐ Update the order menu string on the screen immediately after each revision.

⭐ Once the '+' (ACTIVATE) keycap is pressed, transmit the entered 4-digit order tag to The Things Network through the established LR1302 LoRaWAN gateway connection.

⭐ Notify the user of the LoRa data transfer success on the screen.

⭐ Finally, clear the order menu string after sending it.

⭐ Return to the home screen if the '!' (EXIT) keycap is pressed.

if(active_int_opt[0]){
    show_screen("order_menu", selected_interface_opt, true);
    while(active_int_opt[0]){
      // Get the pressed keycap value depending on the selected keypad map — numbers and letters..
      char activeKey = (current_keypad_map == 1) ? num_keypad.getKey() : let_keypad.getKey();
      if(activeKey != NO_KEY){
        switch(activeKey){
          case '+':
            // Transfer the entered order menu command through the established LoRaWAN network if the '+' (ACTIVATE) keycap is pressed.
            if(order_menu_command.length() == 4){
              adjustColor(0,255,255);
              bool send_status = ra_08h_send_to_lora_gateway(order_menu_command);
              // Notify the user of the LoRa data transfer success on the screen.
              if(send_status){ show_screen("order_menu_success", selected_interface_opt, true); }
              else{ show_screen("order_menu_failure", selected_interface_opt, true); }
              // Clear the order menu command after sending it.
              order_menu_command = "";
            }
          break;
          case '>':
            // Change the keypad to numbers if the '>' (NEXT) keycap is pressed.
            current_keypad_map = 1;
            // Notify the user of the map change success via the screen.
            show_screen("order_menu", selected_interface_opt, false);
          break;
          case '<':
            // Change the keypad to letters if the '<' (PREVIOUS) keycap is pressed.
            current_keypad_map = 2;
            // Notify the user of the map change success via the screen.
            show_screen("order_menu", selected_interface_opt, false);
          break;          
          case '-':
            // Discard one character from the order menu string if the '-' (DELETE) keycap is pressed.
            if(order_menu_command.length() > 0){
              order_menu_command.remove(order_menu_command.length()-1);
              // Show the modified string on the screen.
              show_screen("order_menu", selected_interface_opt, false);
              delay(150);              
            }
          break;
          case '!':
            // Return to the main interface menu if the '!' (EXIT) keycap is pressed.
            active_int_opt[0] = false;
            scr_init = true;
          break;
          case '=':
            // Do nothing if the '=' (SELECT) keycap is pressed.
          break;
          case 'x':
            // Do nothing if an unassigned letter 'x' (X) keycap is pressed.
          break;          
          default:
            // Add the pressed keycap (number or letter) to the order menu string to generate the LoRa command.
            if(order_menu_command.length() < 4){
              order_menu_command += String(activeKey);
              // Show the modified string on the screen.
              show_screen("order_menu", selected_interface_opt, false);
              delay(250);
            }
          break;
        }
      }
    }
  }

⭐ If the add_vehicle endpoint interface option is activated:

⭐ Clear the previously registered values of the image sample collection status array.

⭐ Obtain the pressed keycap value according to the selected keypad map (layout) — numbers or letters.

⭐ If the user did not already authorize a registered vehicle, allow the user to collect image samples to register a new vehicle.

⭐ Select the numbers keypad map once the '>' (NEXT) keycap is pressed.

⭐ Select the letters keypad map once the '<' (PREVIOUS) keycap is pressed.

⭐ Notify the user of the current keypad map selection on the screen.

⭐ When pressed, add the appropriate keycap values (number or letter) to the authentication key string, representing the 4-digit account authentication key assigned by the restaurant web application.

⭐ Discard the latest character from the authentication key string if the '-' (DELETE) keycap is pressed.

⭐ Update the authentication key string on the screen immediately after each revision.

⭐ Once the '+' (ACTIVATE) keycap is pressed, employ the Crowtail serial camera to capture five (5) different image samples of the given vehicle for further object detection model training. Utilize the entered 4-digit account authentication key and the assigned image sample number to create unique JPG image files while saving them to the microSD card.

⭐ On the screen, notify the user of the success of the image-capturing process for each image sample.

⭐ Finally, clear the authentication key string after completing the image sample collection procedure for the new vehicle.

⭐ Return to the home screen if the '!' (EXIT) keycap is pressed.

if(active_int_opt[2]){
    show_screen("add_vehicle", selected_interface_opt, true);
    // Clear the image sample status is previously registered.
    for(int i = 0; i < IMG_SAMPLE_NUM; i++){
      img_save_status[i] = 'w';
    }
    while(active_int_opt[2]){
      // Get the pressed keycap value depending on the selected keypad map — numbers and letters.
      char activeKey = (current_keypad_map == 1) ? num_keypad.getKey() : let_keypad.getKey();
      if(activeKey != NO_KEY){
        // Enable adding a new vehicle if the user did not authorize a registered vehicle.
        if(!vehicle_validation_status){
          switch(activeKey){
            case '+':
              // Obtain the account authentication key (generated by the web dashboard) if the '+' (ACTIVATE) keycap is pressed.
              if(account_auth_key.length() == 4){
                // Activate the image sample collection procedure for the new vehicle.
                show_screen("add_vehicle_img_collect", selected_interface_opt, true);
                delay(500);
                // Capture and save the required vehicle image samples — 5 — for further model training. 
                for(int i = 0; i < IMG_SAMPLE_NUM; i++){
                  capture_and_save_picture(account_auth_key, i);
                  // Notify the user of the status of the image capturing process.
                  show_screen("add_vehicle_img_collect", selected_interface_opt, false);
                  delay(1000);
                }
                // Clear the account authentication key after saving image samples.
                account_auth_key = "";
              }
            break;
            case '>':
              // Change the keypad to numbers if the '>' (NEXT) keycap is pressed.
              current_keypad_map = 1;
              // Notify the user of the map change success via the screen.
              show_screen("add_vehicle", selected_interface_opt, false);
            break;
            case '<':
              // Change the keypad to letters if the '<' (PREVIOUS) keycap is pressed.
              current_keypad_map = 2;
              // Notify the user of the map change success via the screen.
              show_screen("add_vehicle", selected_interface_opt, false);
            break;          
            case '-':
              // Discard one character from the authentication key string if the '-' (DELETE) keycap is pressed.
              if(account_auth_key.length() > 0){
                account_auth_key.remove(account_auth_key.length()-1);
                // Show the modified string on the screen.
                show_screen("add_vehicle", selected_interface_opt, false);
                delay(150);              
              }
            break;
            case '!':
              // Return to the main interface menu if the '!' (EXIT) keycap is pressed.
              active_int_opt[2] = false;
              scr_init = true;
            break;
            case '=':
              // Do nothing if the '=' (SELECT) keycap is pressed.
            break;
            case 'x':
              // Do nothing if an unassigned letter 'x' (X) keycap is pressed.
            break;          
            default:
              // Add the pressed keycap (number or letter) to the authentication key string.
              if(account_auth_key.length() < 4){
                account_auth_key += String(activeKey);
                // Show the modified string on the screen.
                show_screen("add_vehicle", selected_interface_opt, false);
                delay(250);
              }
            break;
          }
        }else{
          switch(activeKey){
            case '!':
              // Return to the main interface menu if the '!' (EXIT) keycap is pressed.
              active_int_opt[2] = false;
              scr_init = true;
            break;
            default:
              // Do nothing is a keycap without an assigned task is pressed.
            break;                        
          }
        }
      }
    }
  }

⭐ If the reset_con endpoint interface option is activated:

⭐ Obtain the pressed keycap value according to the selected keypad map (layout) — numbers or letters.

⭐ Once the '+' (ACTIVATE) keycap is pressed, configure the RA-08H LoRaWAN node module connection settings based on the provided TTN-registered end device information — DEVEUI, JOINEUI (formerly called APPEUI), and APPKEY.

⭐ On the screen, inform the user of the update status of each LoRaWAN connection setting according to the AT command responses.

⭐ Return to the home screen if the '!' (EXIT) keycap is pressed.

if(active_int_opt[3]){
    show_screen("reset_con", selected_interface_opt, true);
    while(active_int_opt[3]){
      // Get the pressed keycap value depending on the selected keypad map — numbers and letters..
      char activeKey = (current_keypad_map == 1) ? num_keypad.getKey() : let_keypad.getKey();
      if(activeKey != NO_KEY){
        switch(activeKey){
          case '+':
            // If requested, configure the RA-08H LoRa module connection settings based on the provided TTN server application.
            if(ttn_settings.ra_08h_update_init){
              configure_ra_08h_lora_settings(ttn_settings.DEVEUI, ttn_settings.APPEUI, ttn_settings.APPKEY);
              // Notify the user of the LoRa settings update status on the screen.
              show_screen("reset_con", selected_interface_opt, false);
            } 
          break;
          case '!':
            // Return to the main interface menu if the '!' (EXIT) keycap is pressed.
            active_int_opt[3] = false;
            scr_init = true;
          break;
          default:
            // Do nothing is a keycap without an assigned task is pressed.
          break;
        }
      }
    }
  }

Step 16.1: Setting the kiosk customer endpoint MicroSD card directory and file structure

#️⃣ Since I decided to utilize custom icons and images to create the kiosk customer endpoint interface menus and system notifications, I uploaded the required visual assets to the microSD card to access and process them via the microSD card slot integrated into the Fermion TFT display.

#️⃣ Also, as mentioned earlier, I created a folder for the collected vehicle image samples.

• assets
• vehicle_image_samples

#️⃣ Then, I tested the kiosk endpoint interface features and responsiveness. Since the RA-08H LoRaWAN node board does not expose a VIN pin, I needed to power the board via USB (powerbank). Thus, the 5V output of the ATX adapter board powers the Crowtail serial camera and the Fermion TFT display since these components are too power-demanding for the node board.

Restaurant Features: Collecting vehicle images and assigning them to a customer account for opt-in vehicle detection

🍔 🚗 🛎️ 📶 The kiosk customer endpoint welcomes customers to the drive-through restaurant by displaying the endpoint home screen, showcasing the available interface options.

• Order Menu
• Validate Vehicle
• Add New Vehicle
• Reset Connection

🍔 🚗 🛎️ 📶 The kiosk customer endpoint allows customers to manage the endpoint interface and enter the necessary account and order information via the custom 4x4 keypad, supporting two different keypad maps (layouts) in accordance with 16 digits of hexadecimal numbers — numbers and letters.

• (1 / A) (2 / B) (3 / C) (>)
• (4 / D) (5 / E) (6 / F) (<)
• (7 / X) (8 / X) (9 / X) (=)
• (+) (0 / X) (-) (!)

🍔 🚗 🛎️ 📶 The kiosk customer endpoint lets customers change the highlighted kiosk endpoint option via the '>' (NEXT) and '<' (PREVIOUS) keycaps.

🍔 🚗 🛎️ 📶 Then, select (activate) the highlighted option via the '=' (SELECT) keycap.

🍔 🚗 🛎️ 📶 Once the Add New Vehicle endpoint interface option is activated, the kiosk endpoint checks the vehicle authorization status. If the customer has already authorized a registered vehicle, the kiosk endpoint notifies the customer on the screen and locks the image sample collection procedure for a new vehicle.

🍔 🚗 🛎️ 📶 Otherwise, the kiosk endpoint lets the customer enter the 4-digit unique account authentication key assigned by the restaurant web application via the custom keypad.

🍔 🚗 🛎️ 📶 While entering the account authentication key, the customer can easily select the numbers keypad map via the '>' (NEXT) keycap and the letters keypad map via the '<' (PREVIOUS) keycap.

🍔 🚗 🛎️ 📶 Once the current keypad map (layout) is changed, the kiosk endpoint notifies the customer on the screen accordingly.

• K: 123
• K: ABC

🍔 🚗 🛎️ 📶 By pressing the '-' (DELETE) keycap, the customer can discard the latest character of the entered account key.

🍔 🚗 🛎️ 📶 Once the 4-digit account key is entered successfully, the customer can initiate the vehicle image sample collection procedure by pressing the '+' (ACTIVATE) keycap.

🍔 🚗 🛎️ 📶 Then, the kiosk endpoint collects five different image samples of the given customer vehicle and saves them to the integrated microSD card for further object detection model training.

🍔 🚗 🛎️ 📶 While collecting and saving vehicle image samples, the kiosk endpoint informs the customer of the success of the image-capturing process on the screen for each image sample individually.

🍔 🚗 🛎️ 📶 At any point, the customer can return to the endpoint home screen by pressing the '!' (EXIT) keycap.

🖥️ For debugging, the RA-08H LoRaWAN node board (RP2040) prints progression notifications on the serial monitor.

#️⃣ As discussed earlier, I did not want to build a 1984-esque drive-through restaurant establishment :) Thus, I decided to give customers the autonomy to collect vehicle information to participate in the account authorization by vehicle detection.

#️⃣ Since vehicle detection is an opt-in feature, customers can employ the order menu interface option of the kiosk endpoint (explained in the following steps) to authorize their accounts by transmitting their authentication keys manually.

#️⃣ After creating four different customer accounts on the restaurant web application, for their registered vehicles, I collected image samples of diverse car replicas I specifically purchased for this project :)

#️⃣ After collecting vehicle image samples labeled with the assigned unique account authentication keys, I constructed my vehicle image data set for training the object detection model successfully.

#️⃣ In this regard, the only disadvantage of enabling customers to collect their vehicle images to provide opt-in vehicle detection is that the restaurant workers need to obtain the collected images from the microSD card so as to train the object detection model manually afterwards.

Step 17: Building an object detection model (FOMO) w/ Edge Impulse Enterprise for vehicle detection

Since Edge Impulse provides developer-friendly tools for advanced AI applications and supports almost every development board due to its model deployment options, I decided to utilize Edge Impulse Enterprise to build my object detection model. Also, Edge Impulse Enterprise incorporates elaborate model architectures for advanced computer vision applications and optimizes the state-of-the-art vision models for edge devices and single-board computers such as the onboard RP2040 of RA-08H LoRaWAN node board.

Among the diverse machine learning algorithms provided by Edge Impulse, I decided to employ FOMO (Faster Objects, More Objects) since it is a groundbreaking algorithm optimized for both highly constrained edge devices and powerful single-board computers.

While labeling the vehicle image samples, I simply applied the unique account authentication keys produced by the restaurant web application:

• a1dd
• a2f9
• a4c7
• a9f1

Plausibly, Edge Impulse Enterprise enables developers with advanced tools to build, optimize, and deploy each available machine learning algorithm as supported firmware for nearly any device you can think of. Therefore, after training and validating, I was able to deploy my FOMO model as an Arduino library compatible with RP2040.

To utilize the advanced AI tools provided by Edge Impulse, you can register here.

Furthermore, you can inspect this FOMO object detection model on Edge Impulse as a public project.

Step 17.1: Uploading and labeling training and testing vehicle images (samples)

#️⃣ First, I created a new project on my Edge Impulse Enterprise account.

#️⃣ To employ the bounding box labeling tool for object detection models, I navigated to Dashboard ➡ Project info ➡ Labeling method and selected Bounding boxes (object detection).

#️⃣ To upload training and testing vehicle image samples as individual files, I opened the Data acquisition section and clicked the Upload data icon.

#️⃣ Then, I navigated to Data acquisition ➡ Labeling queue to access all unlabeled items (training and testing) remaining in the provided image data set.

#️⃣ After drawing bounding boxes around target objects, I clicked the Save labels button to complete labeling an image sample. Then, I repeated this process until all vehicle image samples retained at least one labeled target object.

Step 17.2: Training the FOMO model on the vehicle images

An impulse (an application developed and optimized by Edge Impulse) takes raw data, applies signal processing to extract features, and then utilizes a learning block to classify new data.

For my application, I created the impulse by employing the Image processing block and the Object Detection (Images) learning block.

Image processing block processes the passed raw image input as grayscale or RGB (optional) to produce a reliable features array.

Object Detection (Images) learning block represents the officially supported machine learning algorithms performing object detection.

#️⃣ First, I opened the Impulse design ➡ Create impulse section, set the model image resolution to 96 x 96, and selected the Fit shortest axis resize mode so as to scale (resize) the given image samples precisely. To complete the impulse creation, I clicked Save Impulse.

#️⃣ To modify the raw image features in the applicable format, I navigated to the Impulse design ➡ Image section, set the Color depth parameter as RGB, and clicked Save parameters.

#️⃣ Then, I proceeded to click Generate features to extract the required features for training by applying the Image processing block.

#️⃣ After extracting features successfully, I navigated to the Impulse design ➡ Object detection section and modified the neural network settings and architecture to achieve reliable accuracy and validity.

#️⃣ According to my prolonged experiments, I assigned the final model settings as follows.

📌 Neural network settings:

• Number of training cycles ➡ 75
• Learning rate ➡ 0.035
• Validation set size ➡ 5%

📌 Neural network architecture:

• FOMO (Faster Objects, More Objects) MobileNetV2 0.35

#️⃣ After training the model with the final configurations, Edge Impulse evaluated the F1 score (accuracy) as 100.0% due to the modest volume of the validation set.

Step 17.3: Evaluating the model accuracy and deploying the validated model

#️⃣ First, to obtain the validation score of the trained model based on the provided testing samples, I navigated to the Impulse design ➡ Model testing section and clicked Classify all.

#️⃣ After executing the model testing process, Edge Impulse evaluated the model accuracy (precision) as 100.00%.

#️⃣ To deploy the validated model optimized for my hardware, I navigated to the Impulse design ➡ Deployment section and searched for Arduino library.

#️⃣ I chose the Quantized (int8) optimization to achieve the optimal performance when running the deployed model.

#️⃣ Finally, I clicked Build to download the produced Arduino library, containing the trained object detection model.

Step 18: Initial programming of Arduino Nicla Vision

As discussed earlier, the programming process was not linear, and the code snippets described below showcase only a limited aspect of the presented code file. Thus, please refer to the project GitHub page to review code files thoroughly.

📁 server_secrets.h

⭐ Define the required Wi-Fi network credentials and the webhook information for communicating with the restaurant web application hosted by LattePanda Mu (N305).

char ssid[] = "<SSID>";          // your network SSID (name)
char pass[] = "<PASSWORD>";      // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;                // your network key Index number (needed only for WEP)

// Define the required webhook information, hosted by LattePanda Mu (N305).
char server[] = "192.168.1.21";
String application = "/ai_driven_lora_kiosk/assets/food_delivery_system_conn.php";

📁 ai_driven_customer_service_food_delivery_system.ino

⭐ Include the required libraries.

#include <WiFi.h>
#include "camera.h"
#include "gc2145.h"
#include <ea_malloc.h>
#include "VL53L1X.h"

⭐ Include the provided restaurant web application webhook information and initialize the WiFiClient object.

#include "server_secrets.h"

// Initialize the WiFiClient object.
WiFiClient client; /* WiFiSSLClient client; */

⭐ Define the required class objects for controlling the onboard 2-megapixel CMOS camera (GC2145). Then, declare the camera frame buffer.

GC2145 galaxyCore;
Camera cam(galaxyCore);

// Define the camera frame buffer.
FrameBuffer fb;

⭐ Define the onboard Time of Flight (ToF) distance sensor (VL53L1CBV0FY) object.

VL53L1X proximity;

⭐ By creating a struct, define the required variables for controlling three stepper motors, including DIR and STEP pin duos utilized by the A4988 driver modules.

struct stepper_config{
  #define m_num 3
  PinName _pins[m_num][2] = {{PE_12, PE_13}, {PE_14, PE_11}, {PG_12, PB_9}}; // (DIR, STEP)
  // Assign the required revolution and speed variables based on axes — X, Y, and Z (arm).
  int stepsPerRevolution = 200;
  int x_y_speed = 16000;
  int z_speed = 12000;
  // Define stepper motor homing configurations.
  int home_step_number[m_num] = {1, 1, 3};
  // Assign stepper motor tasks based on the associated part.
  int x_core = 0, y_core = 1, z_arm = 2;
  // Assign the required preperation time per requested food item.
  int wait_per_food_item = 3000;
  // Assign the required time for customers to get all of the food items from the tray.
  int customer_wait_time = 10000;
};

⭐ Activate the assigned DIR and STEP pins connected to the A4988 driver modules.

for(int i = 0; i < m_num; i++){ pinMode(stepper_config._pins[i][0], OUTPUT); pinMode(stepper_config._pins[i][1], OUTPUT); }

⭐ Attempt to connect to the Wi-Fi network with the provided credentials. Then, wait until the connection is established.

WiFi.begin(ssid, pass);
  // Attempt to connect to the Wi-Fi network:
  while(WiFi.status() != WL_CONNECTED){
    // Wait for the connection:
    delay(500);
    Serial.print(".");
  }
  // If connected to the network successfully:
  Serial.println("Connected to the Wi-Fi network successfully!");

⭐ Initialize the GC2145 camera with the required pixel format, FPS, and resolution configurations.

❗ While collecting the AprilTag sign image samples, I utilized the 320 x 320 resolution to obtain a broader view. Nonetheless, I needed to reduce the resolution to 320 x 240 while running inferences with the object detection model to avoid tensor allocation and memory issues.

if(!cam.begin(CAMERA_R320x240, CAMERA_RGB565, 30)) { // CAMERA_R320x240 [model], CAMERA_R320x320 [samples]
    Serial.println("GC2145 camera: initialization failed!");
  }else{
    Serial.println("GC2145 camera: initialized successfully!");
  }

⭐ Define the required settings to initialize the onboard VL53L1X Time of Flight (ToF) distance sensor — 400 kHz I2C. Then, declare its data reading configurations.

Wire1.begin();
  Wire1.setClock(400000); // use 400 kHz I2C
  proximity.setBus(&Wire1);
  if(!proximity.init()) { Serial.println("ToF sensor: initialization failed!"); while (1); }
  else{ Serial.println("ToF sensor: initialized successfully!"); }
  // Adjust ToF sensor data reading configurations.
  proximity.setDistanceMode(VL53L1X::Long);
  proximity.setMeasurementTimingBudget(50000);
  proximity.startContinuous(50);

⭐ In the web_app_conn_channel function:

⭐ After connecting to the restaurant web application successfully, perform different operations according to the passed connection mode.

⭐ If the passed connection mode is listen:

⭐ Create the required query string to make an HTTP GET request to the provided webhook so as to obtain the latest still ongoing food delivery system log information.

⭐ After obtaining the log information successfully, when the food delivery system is not idle, modify the received string to extract the data packet (log) from the HTTP header.

⭐ Finally, return the extracted log information for further analysis.

⭐ If the passed connection mode is update:

⭐ Create the required query string to make an HTTP GET request to the provided webhook so as to update the task status of the latest ongoing food delivery system log.

⭐ After completing this request, stop the client connection to prevent errors since the listen connection mode is running continuously in the loop.

⭐ If the passed connection mode is send_img:

⭐ Create the query string to enable making an HTTP POST request to the provided webhook to transfer the captured raw image buffer.

⭐ After declaring the boundary parameter containing the captured image buffer, estimate the total message length and initiate the POST request.

⭐ Wait until transferring the passed image buffer. Then, stop the client connection to prevent errors.

String web_app_conn_channel(String conn_mode, String _info){
  String app_res = "";
  // Connect to the given web application. Change '80' with '443' if you are using SSL connection.
  if(client.connect(server, 80)){
    // If the connection is established successfully, perform the passed connection mode.
    Serial.println("\nConnected to the web application successfully!\n");  
    if(conn_mode == "listen"){
      // Create the query string accordingly.
      String query = application + "?get_latest_food_system_log";  
      // Make an HTTP GET request to the provided webhook to optain latest commands.
      client.println("GET " + query + " HTTP/1.1");
      client.print("Host: "); client.println(server);
      client.println("Connection: close");
      client.println();
      // After performing the requested mode successfully, obtain the application response.
      delay(1000);
      while (client.available()) app_res = client.readString();
      // Then, modify the string to derive the data packet from the header when the delivery system is not idle.
      if(app_res.indexOf("%") >= 0){
        app_res = app_res.substring(app_res.indexOf("!") + 1, app_res.indexOf("!", app_res.indexOf("!") + 1));
      }else{
        app_res = "idle";
        Serial.println("\nFood delivery system is idle!\n");
      }           
    }else if(conn_mode == "update"){
      // Create the query string accordingly.
      String query = application + "?update_latest_food_system_log_status=" + _info;  
      // Make an HTTP GET request to the provided webhook to update the food system log status.
      client.println("GET " + query + " HTTP/1.1");
      client.print("Host: "); client.println(server);
      client.println("Connection: close");
      client.println();
      delay(3000);
      // After updating, stop the client connection to prevent errors.
      client.stop();
      Serial.println("\nOrder status is updated successfully! \n");      
    }else if(conn_mode == "send_img"){
      // Create the query string accordingly.
      String query = application + "?save_slot_image_sample=" + _info;     
      // Make an HTTP POST request to transfer the captured image buffer.
      String head = "--SlotSample\r\nContent-Disposition: form-data; name=\"captured_image\"; filename=\"slot_sample.txt\"\r\nContent-Type: text/plain\r\n\r\n";
      String tail = "\r\n--SlotSample--\r\n";
      // Estimate the total message length.
      uint32_t totalLen = head.length() + cam.frameSize() + tail.length();
      // Initiate the POST request.
      client.println("POST " + query + " HTTP/1.1");
      client.print("Host: "); client.println(server);
      client.println("Content-Length: " + String(totalLen));
      client.println("Content-Type: multipart/form-data; boundary=SlotSample");
      client.println();
      client.print(head);
      client.write(fb.getBuffer(), cam.frameSize());
      client.print(tail);
      client.println("Connection: close");
      client.println();
      // Wait until transferring the passed image buffer. Then, stop the client connection to prevent errors.
      Serial.println("\nSlot sample image is transferred!");
      delay(3000);
      client.stop();
    }
  // If the connection is failed:
  }else{
    app_res = "failed";
    Serial.println("\nConnection failed to the web application!\n");
    delay(1000);
  }
  // Finally, return the generated application response for further analysis.
  return app_res;
}

⭐ In the h_bot_mechanism_move function:

⭐ Move the stepper motors controlling the food delivery system gantry head, driven by a single timing belt, according to the H-Bot kinematic structure principles.

void h_bot_mechanism_move(int step_number, int acc, String axis, String move_to){
  /*
    Move the H-Bot mechanism by driving the associated stepper motors according to these calculations:
    Move along X-axis: rotate stepper motors in the same direction at the same velocity.
    Move along Y-axis: rotate stepper motors in opposite directions at the same velocity. 
    Move diagonally: rotate only one of the stepper motors. 
  */
  if(axis == "X"){
    if(move_to == "left"){
      digitalWrite(stepper_config._pins[stepper_config.x_core][0], HIGH);
      digitalWrite(stepper_config._pins[stepper_config.y_core][0], HIGH);      
    }else if(move_to == "right"){
      digitalWrite(stepper_config._pins[stepper_config.x_core][0], LOW);
      digitalWrite(stepper_config._pins[stepper_config.y_core][0], LOW);
    }
  }
  if(axis == "Y"){
    if(move_to == "up"){
      digitalWrite(stepper_config._pins[stepper_config.x_core][0], LOW);
      digitalWrite(stepper_config._pins[stepper_config.y_core][0], HIGH); 
    }else if(move_to == "down"){
      digitalWrite(stepper_config._pins[stepper_config.x_core][0], HIGH);
      digitalWrite(stepper_config._pins[stepper_config.y_core][0], LOW);       
    }
  }

  for(int i = 0; i < step_number; i++){
    digitalWrite(stepper_config._pins[stepper_config.x_core][1], HIGH);
    digitalWrite(stepper_config._pins[stepper_config.y_core][1], HIGH);
    delayMicroseconds(stepper_config.x_y_speed/acc);
    digitalWrite(stepper_config._pins[stepper_config.x_core][1], LOW);
    digitalWrite(stepper_config._pins[stepper_config.y_core][1], LOW);
    delayMicroseconds(stepper_config.x_y_speed/acc);
  }  

}

⭐ In the h_bot_mechanism_home function, when requested, perform the homing sequences for the X-axis and Y-axis by utilizing their associated limit (micro) switches.

void h_bot_mechanism_home(bool x_axis, bool y_axis){
  /* Home the requested axis via the assigned limit (micro) switches. */
  if(x_axis){
    while(digitalRead(micro_switch_B)){
      h_bot_mechanism_move(5, 2, "X", "left");
    }delay(500);
    h_bot_mechanism_move(30, 10, "X", "right");
    delay(500);
  }
  if(y_axis){
    while(digitalRead(micro_switch_A)){
      h_bot_mechanism_move(5, 2, "Y", "down");
    }delay(500);
    h_bot_mechanism_move(30, 10, "Y", "up");
    delay(500);
  }  
}

⭐ In the z_arm_move function, rotate the stepper motor controlling the food carrier arm (tray) clockwise or counterclockwise.

void z_arm_move(int step_number, int acc, String _dir){
  /*
    Move the arm carrying the tray.
    CW:  Clockwise
    CCW: Counter-clockwise
  */
  if(_dir == "CW"){ digitalWrite(stepper_config._pins[stepper_config.z_arm][0], HIGH); }
  else if(_dir == "CCW"){ digitalWrite(stepper_config._pins[stepper_config.z_arm][0], LOW); }
	
	for(int i = 0; i < step_number; i++){
	  digitalWrite(stepper_config._pins[stepper_config.z_arm][1], HIGH);
		delayMicroseconds(stepper_config.z_speed/acc);
		digitalWrite(stepper_config._pins[stepper_config.z_arm][1], LOW);
		delayMicroseconds(stepper_config.z_speed/acc);
	}
}

⭐ In the take_picture function, capture the latest frame (image) produced by the built-in GC2145 camera.

void take_picture(){
  // Capture a picture with the GC2145 camera.
  if(cam.grabFrame(fb, 3000) == 0){
   Serial.println("\nGC2145 camera: image captured successfully!");
  }else{
    Serial.println("\nGC2145 camera: image capture failed!");
  }
  delay(2000);
}

⭐ In the get_tof_sensor_data function, obtain the latest distance measurement from the onboard ToF sensor.

int get_tof_sensor_data(){
  // Obtain the most recent ToF sensor reading.
  return proximity.read();
}

#️⃣ Even though the food delivery system operations are controlled by the restaurant web application, I utilized the control buttons to test the stepper motor movements for fine-tuning and collect AprilTag sign sample images manually.

⭐ Once the control button D is pressed, change the current manual task number. When activated, each manual task is distinguished with a different color via the onboard RGB LED.

⭐ If the manual task [0] is activated:

⭐ Initiate the X-axis homing sequence once the control button A is pressed.

⭐ Initiate the Y-axis homing sequence once the control button B is pressed.

⭐ If the manual task [1] is activated:

⭐ Move the gantry head one step right once the control button A is pressed.

⭐ Move the gantry head one step left once the control button B is pressed.

⭐ Also, check the limit (micro) switch pin states via the serial monitor.

⭐ If the manual task [2] is activated:

⭐ Move the gantry head one step up once the control button A is pressed.

⭐ Move the gantry head one step down once the control button B is pressed.

⭐ If the manual task [3] is activated:

⭐ Rotate the food carrier arm clockwise once the control button A is pressed.

⭐ Rotate the food carrier arm counterclockwise once the control button B is pressed.

⭐ If the manual task [4] is activated:

⭐ Capture an AprilTag sign image via the onboard GC2145 camera, assigned to a food prep station.

⭐ Then, transfer the captured raw image buffer to the restaurant web application.

if(!digitalRead(control_button_D)){
    current_manual_task++;
    if(current_manual_task > 4) current_manual_task = 0;
    delay(1000);
  }
  switch(current_manual_task){
    // Home X-axis and Y-axis (manually)
    case 0:
      adjustColor(0,1,1);
      if(!digitalRead(control_button_A)) h_bot_mechanism_home(true, false);
      if(!digitalRead(control_button_B)) h_bot_mechanism_home(false, true);
    break;
    // Move X-axis (manually)
    case 1:
      adjustColor(1,0,0);
      if(!digitalRead(control_button_A)) h_bot_mechanism_move(stepper_config.stepsPerRevolution/2, 10, "X", "right");
      if(!digitalRead(control_button_B)) h_bot_mechanism_move(stepper_config.stepsPerRevolution/2, 10, "X", "left");
      /* Debug micro switch pin responses. */
      if(!digitalRead(micro_switch_A)) Serial.println("Limit Switch [A] => Working!\n");
      if(!digitalRead(micro_switch_B)) Serial.println("Limit Switch [B] => Working!\n");
    break;
    // Move Y-axis (manually)
    case 2:
      adjustColor(0,1,0);
      if(!digitalRead(control_button_A)) h_bot_mechanism_move(stepper_config.stepsPerRevolution/2, 10, "Y", "up");
      if(!digitalRead(control_button_B)) h_bot_mechanism_move(stepper_config.stepsPerRevolution/2, 10, "Y", "down");
    break;
    // Move the arm (manually)
    case 3:
      adjustColor(0,0,1);
      if(!digitalRead(control_button_A)) z_arm_move(stepper_config.stepsPerRevolution/2, 10, "CW");
      if(!digitalRead(control_button_B)) z_arm_move(stepper_config.stepsPerRevolution/2, 10, "CCW");
    break;
    // Capture prep station AprilTag (unique for each station) sample images and transfer the captured AprilTag pictures to the given web application.
    case 4:
      adjustColor(1,0,1);
      if(!digitalRead(control_button_A)){
        take_picture();
        web_app_conn_channel("send_img", "station_tag_img_");
      }
    break;              
    default:
      // Do nothing.
    break;
  }

Step 18.1: H-Bot-inspired food delivery mechanism fine-tuning and adjustments

#️⃣ After programming Nicla Vision, I connected the Nema 17 stepper motors to the food delivery system Flex PCB and started testing the food delivery system features rigorously to review the connection status between the restaurant web application and Nicla Vision.

#️⃣ Before attaching the delivery system Flex PCB to the gantry head, I evaluated the required step numbers for repeatable axes homing sequences.

#️⃣ After concluding initial testing, I attached the Flex PCB to its pliable case on the gantry head via M3 screws through M3 standoffs.

#️⃣ After cable-managing the stepper motor connections, I analysed the H-Bot-inspired food delivery system gantry head movements for a while to fine-tune the pulley system to avoid jerks and unbalanced torque.

#️⃣ I realized that the single GT2 timing belt skips teeth due to thermal expansion. Thus, I added two torsion springs to tension the timing belt.

#️⃣ I also noticed that the food carrier arm rarely bumps into the main frame edges due to the inclination of my table. Thus, I fastened a ball caster wheel near the stepper motor controlling the carrier arm.

#️⃣ I also strengthened the stepper motor lid connections via a hot glue gun to prevent oscillations.

Restaurant Features: Controlling the food delivery system gantry head movements manually

#️⃣ As discussed earlier, even though the restaurant web application manages the H-Bot-inspired food delivery system movements, I enabled manual tasks via the integrated control buttons on the Flex PCB to test the stepper motor movements and axes homing sequences for fine-tuning and optimization.

🍔 🚗 🛎️ 📶 The food delivery system allows the user to change the active manual task by pressing the control button D. After reaching the last manual task number, the system returns to the default manual task — 0.

🍔 🚗 🛎️ 📶 The manual task [0] is the default and lets the user initiate the X-axis homing sequence by pressing the control button A and the Y-axis homing sequence by pressing the control button B. It turns the onboard RGB LED of Nicla Vision to cyan once activated.

🍔 🚗 🛎️ 📶 Once the manual task [1] is activated, it lets the user move the gantry head to the right by pressing the control button A and to the left by pressing the control button B. It turns the onboard RGB LED to red.

🍔 🚗 🛎️ 📶 Once the manual task [2] is activated, it lets the user move the gantry head up by pressing the control button A and down by pressing the control button B. It turns the onboard RGB LED to green.

🍔 🚗 🛎️ 📶 Once the manual task [3] is activated, it lets the user rotate the food carrier arm clockwise by pressing the control button A and counterclockwise by pressing the control button B. It turns the onboard RGB LED to blue.

Restaurant Features: Collecting AprilTag sign images for the food prep station identification

🍔 🚗 🛎️ 📶 By applying the previous manual tasks, the food delivery system allows the user to control the gantry head movements to align the onboard GC2145 camera of Nicla Vision and the food prep station AptilTag signs facing each other.

🍔 🚗 🛎️ 📶 Then, once the manual task [4] is activated, it lets the user capture a picture of the targeted AprilTag sign and transfer the captured raw image buffer to the restaurant web application via an HTTP POST request. It turns the onboard RGB LED to magenta.

🍔 🚗 🛎️ 📶 After obtaining the captured raw image buffers, the restaurant web application saves them as text (TXT) files.

#️⃣ Nonetheless, as discussed earlier, I decided to employ a simple Python script to convert all of the raw image buffers to JPG files since I did not want to consume network bandwidth while converting buffers automatically via the web application.

#️⃣ To be able to execute my Python script, I needed to download the pip package installer and install the NumPy package.

#️⃣ Since I did not create a virtual environment, I needed to utilize the break-system-packages command-line argument to bypass the system-wide package installation error.

sudo apt install python3-pip

pip install numpy --break-system-packages

#️⃣ First, I navigated to the folder containing the Python script and then executed it via the terminal.

cd /opt/lampp/htdocs/ai_driven_lora_kiosk/assets/prep_stations/apriltag_samples

python3 RGB565_raw_buffer_converter.py

🍔 🚗 🛎️ 📶 Once the Python script is executed, the restaurant web application converts the raw image buffers to JPG files and saves the converted images to the jpg_converted folder for further object detection model training.

🖥️ For debugging, Arduino Nicla Vision prints progression notifications on the serial monitor.

#️⃣ After collecting image samples of the unique AprilTag signs assigned to six different food prep stations, I constructed my AprilTag sign data set for training the object detection model successfully.

Step 19: Building an object detection model (FOMO) w/ Edge Impulse Enterprise for food prep station identification

Since Edge Impulse provides developer-friendly tools for advanced AI applications and supports almost every development board due to its model deployment options, I decided to utilize Edge Impulse Enterprise to build my object detection model. Also, Edge Impulse Enterprise incorporates elaborate model architectures for advanced computer vision applications and optimizes the state-of-the-art vision models for edge devices and single-board computers such as Arduino Nicla Vision.

Among the diverse machine learning algorithms provided by Edge Impulse, I decided to employ FOMO (Faster Objects, More Objects) since it is a groundbreaking algorithm optimized for both highly constrained edge devices and powerful single-board computers.

While labeling the AprilTag sign image samples, I simply applied the food prep station numbers to which they are assigned:

• station_1
• station_2
• station_3
• station_4
• station_5
• station_6

Plausibly, Edge Impulse Enterprise enables developers with advanced tools to build, optimize, and deploy each available machine learning algorithm as supported firmware for nearly any device you can think of. Therefore, after training and validating, I was able to deploy my FOMO model as an Arduino library compatible with Nicla Vision.

To utilize the advanced AI tools provided by Edge Impulse, you can register here.

Furthermore, you can inspect this FOMO object detection model on Edge Impulse as a public project.

Step 19.1: Uploading and labeling training and testing AprilTag sign images (samples)

#️⃣ First, I created a new project on my Edge Impulse Enterprise account.

#️⃣ To employ the bounding box labeling tool for object detection models, I navigated to Dashboard ➡ Project info ➡ Labeling method and selected Bounding boxes (object detection).

#️⃣ To upload training and testing AprilTag sign image samples as individual files, I opened the Data acquisition section and clicked the Upload data icon.

#️⃣ Then, I navigated to Data acquisition ➡ Labeling queue to access all unlabeled items (training and testing) remaining in the provided image data set.

#️⃣ After drawing bounding boxes around target objects, I clicked the Save labels button to complete labeling an image sample. Then, I repeated this process until all AprilTag sign image samples retained at least one labeled target object.

Step 19.2: Training the FOMO model on unique AprilTag sign images

An impulse (an application developed and optimized by Edge Impulse) takes raw data, applies signal processing to extract features, and then utilizes a learning block to classify new data.

For my application, I created the impulse by employing the Image processing block and the Object Detection (Images) learning block.

Image processing block processes the passed raw image input as grayscale or RGB (optional) to produce a reliable features array.

Object Detection (Images) learning block represents the officially supported machine learning algorithms performing object detection.

#️⃣ First, I opened the Impulse design ➡ Create impulse section, set the model image resolution to 48 x 48, and selected the Fit shortest axis resize mode so as to scale (resize) the given image samples precisely. To complete the impulse creation, I clicked Save Impulse.

#️⃣ To modify the raw image features in the applicable format, I navigated to the Impulse design ➡ Image section, set the Color depth parameter as RGB, and clicked Save parameters.

#️⃣ Then, I proceeded to click Generate features to extract the required features for training by applying the Image processing block.

#️⃣ After extracting features successfully, I navigated to the Impulse design ➡ Object detection section and modified the neural network settings and architecture to achieve reliable accuracy and validity.

#️⃣ According to my prolonged experiments, I assigned the final model settings as follows.

📌 Neural network settings:

• Number of training cycles ➡ 60
• Learning rate ➡ 0.015
• Validation set size ➡ 5%

📌 Neural network architecture:

• FOMO (Faster Objects, More Objects) MobileNetV2 0.35

#️⃣ After training the model with the final configurations, Edge Impulse evaluated the F1 score (accuracy) as 100.0% due to the modest volume of the validation set.

#️⃣ Before configuring the 48 x 48 model image resolution, I was going to utilize higher resolutions to train a more accurate model. Unfortunately, I encountered tensor allocation and memory issues once I deployed the trained model on Nicla Vision.

#️⃣ Thanks to the built-in Experiments feature, I was able to create different impulses with varying configurations and test each version on the same data set to review their performances.

Step 19.3: Evaluating the model accuracy and deploying the validated model

#️⃣ First, to obtain the validation score of the trained model based on the provided testing samples, I navigated to the Impulse design ➡ Model testing section and clicked Classify all.

#️⃣ After executing the model testing process, Edge Impulse evaluated the model accuracy (precision) as 83.33%.

#️⃣ To deploy the validated model optimized for my hardware, I navigated to the Impulse design ➡ Deployment section and searched for Arduino library.

#️⃣ I chose the Quantized (int8) optimization to achieve the optimal performance when running the deployed model.

#️⃣ Finally, I clicked Build to download the produced Arduino library, containing the trained object detection model.

Step 20: Setting up the FOMO object detection model (vehicle) on RA-08H LoRaWAN node board

#️⃣ After downloading the trained model as an Arduino library in ZIP file format, I needed to enable the installed library in the Arduino IDE for deployment on the onboard RP2040 of RA-08H LoRaWAN node board.

#️⃣ Thus, I navigated to Sketch ➡ Include Library ➡ Add.ZIP Library... to enable the Arduino library containing the trained and optimized model.

📁 ai_driven_customer_service_kiosk_lora_end_point.ino

⭐ Include the Edge Impulse FOMO model converted to an Arduino library and the necessary built-in Edge Impulse image (frame) processing functions.

#include <LoRa_LLM-enabled_Drive-through_Kiosk_Vehicle_Detection_inferencing.h>
#include "edge-impulse-sdk/dsp/image/image.hpp"

⭐ Then, define the required parameters to run an inference with the provided Edge Impulse FOMO model.

#define CAPTURED_IMAGE_BUFFER_COLS    320
#define CAPTURED_IMAGE_BUFFER_ROWS    240
#define EI_CAMERA_FRAME_BYTE_SIZE     3
uint8_t *ei_camera_capture_out;

⭐ Define the individual class (label) names provided while training this FOMO model.

int classes_item_num = 4;
String classes[] = {"a1dd", "a2f9", "a4c7", "a9f1"};

⭐ In the RBG565ToRGB888 function, convert the passed raw RGB565 image buffer to an RGB888 buffer.

bool RBG565ToRGB888(uint8_t *src_buf, uint8_t *dst_buf, uint32_t src_len){
    uint8_t hb, lb;
    uint32_t pix_count = src_len / 2;
    // Convert the passed raw RGB565 image buffer to an RGB888 buffer.
    for(uint32_t i = 0; i < pix_count; i ++) {
       // Next source byte.
        hb = *src_buf++;
        lb = *src_buf++;
        // Conversion [RGB565 -> RGB888].
        *dst_buf++ = hb & 0xF8;
        *dst_buf++ = (hb & 0x07) << 5 | (lb & 0xE0) >> 3;
        *dst_buf++ = (lb & 0x1F) << 3;
    }
    return true;
}

⭐ In the ei_camera_cutout_get_data function:

⭐ Convert the provided image data (buffer) to the out_ptr format required by the Edge Impulse FOMO model.

⭐ Considering the passed image buffer is already converted to RGB888, recalculate the required offset into pixel index (array).

static int ei_camera_cutout_get_data(size_t offset, size_t length, float *out_ptr){
  // Convert the given image data (buffer) to the out_ptr format required by the Edge Impulse FOMO model.
  size_t pixel_ix = offset * 3;
  size_t pixels_left = length;
  size_t out_ptr_ix = 0;
  // Since the image data is already converted to an RGB888 buffer, directly recalculate offset into pixel index.
  while(pixels_left != 0){
    out_ptr[out_ptr_ix] = (ei_camera_capture_out[pixel_ix] << 16) + (ei_camera_capture_out[pixel_ix + 1] << 8) + ei_camera_capture_out[pixel_ix + 2];
    // Move to the next pixel.
    out_ptr_ix++;
    pixel_ix+=3;
    pixels_left--;
  }
  return 0;
}

⭐ In the run_inference function:

⭐ Summarize the Edge Impulse FOMO model inference settings, provided by the model_metadata.h file, and print the summary on the serial monitor.

⭐ Capture a snapshot (image frame) from the video stream produced by the Crowtail serial camera (VC0706).

⭐ Convert the captured raw RGB565 image buffer to an RGB888 buffer.

⭐ Depending on the given model's image (frame) resolution, resize the converted RGB888 buffer by utilizing built-in Edge Impulse functions.

⭐ Create a signal object from the converted and resized RGB888 image buffer.

⭐ Then, run an inference with the provided classifier to make predictions based on the trained classes.

⭐ Print the inference timings on the serial monitor.

⭐ If the model detects an object successfully, obtain labels (classes) and bounding box measurements for each detected target object on the given image buffer.

⭐ Print the model detection results and the calculated bounding box measurements on the serial monitor.

⭐ If the detected label is in the provided class array, update the inference status to detected.

❗As mentioned earlier, the restaurant workers need to train the model manually while adding new customer vehicles (opt-in). Thus, I made adding the new labels (account authentication keys) to the integrated class array after training the model a necessity to ensure the kiosk customer endpoint is always up to date.

⭐ If the model cannot detect any objects, update the inference status to none.

⭐ Print inference anomalies on the serial monitor, if any.

⭐ Release the image buffer processed by the signal object.

⭐ Reset the Crowtail serial camera (VC0706) after taking a snapshot to clear the integrated image buffer.

void run_inference(){
  adjustColor(0,0,255);
  // Summarize the Edge Impulse FOMO model inference settings (from model_metadata.h).
  ei_printf("\nInference settings:\n");
  ei_printf("\tImage resolution: %dx%d\n", EI_CLASSIFIER_INPUT_WIDTH, EI_CLASSIFIER_INPUT_HEIGHT);
  ei_printf("\tFrame size: %d\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
  ei_printf("\tNo. of classes: %d\n", sizeof(ei_classifier_inferencing_categories) / sizeof(ei_classifier_inferencing_categories[0]));

  // Capture a new image (frame) via the Crowtail serial camera (VC0706).
  if(serial_cam.takePicture()){
    // Convert the captured raw RGB565 image buffer to a RGB888 buffer required by the Edge Impulse functions.
    ei_camera_capture_out = (uint8_t*)malloc(CAPTURED_IMAGE_BUFFER_COLS * CAPTURED_IMAGE_BUFFER_ROWS * EI_CAMERA_FRAME_BYTE_SIZE);
    RBG565ToRGB888(serial_cam.readPicture(serial_cam.frameLength()), ei_camera_capture_out, serial_cam.frameLength());
    // Depending on the given model's image (frame) resolution, resize the converted RGB888 buffer by utilizing built-in Edge Impulse functions.
    if(CAPTURED_IMAGE_BUFFER_COLS != EI_CLASSIFIER_INPUT_WIDTH && CAPTURED_IMAGE_BUFFER_ROWS != EI_CLASSIFIER_INPUT_HEIGHT){
      ei::image::processing::crop_and_interpolate_rgb888(
        ei_camera_capture_out, // Output image buffer, can be same as the input image buffer.
        CAPTURED_IMAGE_BUFFER_COLS,
        CAPTURED_IMAGE_BUFFER_ROWS,
        ei_camera_capture_out,
        EI_CLASSIFIER_INPUT_WIDTH,
        EI_CLASSIFIER_INPUT_HEIGHT);
    }
    // Run an inference to make predictions based on the trained classes.
    ei::signal_t signal;
    // Create a signal object from the converted and resized RGB888 image buffer.
    signal.total_length = EI_CLASSIFIER_INPUT_WIDTH * EI_CLASSIFIER_INPUT_HEIGHT;
    signal.get_data = &ei_camera_cutout_get_data;
    // Run the provided classifier.
    ei_impulse_result_t result = { 0 };
    EI_IMPULSE_ERROR _err = run_classifier(&signal, &result, false);
    if(_err != EI_IMPULSE_OK){
      ei_printf("ERR: Failed to run classifier (%d)\n", _err);
      return;
    }

    // Print the inference timings on the serial monitor.
    ei_printf("\nPredictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);

    // Obtain the object detection results and bounding boxes for the detected labels (classes). 
    bool bb_found = result.bounding_boxes[0].value > 0;
    for(size_t ix = 0; ix < EI_CLASSIFIER_OBJECT_DETECTION_COUNT; ix++){
      auto bb = result.bounding_boxes[ix];
      if(bb.value == 0) continue;
      // Print the calculated bounding box measurements on the serial monitor.
      ei_printf("    %s (", bb.label);
      ei_printf_float(bb.value);
      ei_printf(") [ x: %u, y: %u, width: %u, height: %u ]\n", bb.x, bb.y, bb.width, bb.height);
      // Fetch the predicted label (class) and the detected object's bounding box measurements (if necessary).
      for(int i=0; i<classes_item_num; i++){
        if(bb.label == classes[i]){
          detected_class = bb.label;
          inference_status = 'd';
          ei_printf("\nPredicted Class [Label]: %s\n", detected_class); 
        }
      }
    }
    // Notify the user the model cannot detect any labels.
    if(!bb_found){
      ei_printf("\nPredicted Class [Label]: No objects found!\n");
      detected_class = "none";
      inference_status = 'n';
    }
    // Notify the user of the model anomalies if occured any while running the inference.
    #if EI_CLASSIFIER_HAS_ANOMALY == 1
      ei_printf("Anomaly: ");
      ei_printf_float(result.anomaly);
      ei_printf("\n");
    #endif 
    // Release the image buffer processed by the signal object.
    ei_free(ei_camera_capture_out);
    // Reset the Crowtail serial camera (VC0706) after running the inference to clear the image frame.
    serial_cam.reset();    
  }
}

⭐ If the valid_vehicle endpoint interface option is activated:

⭐ Obtain the pressed keycap value according to the selected keypad map (layout) — numbers or letters.

⭐ Once the '+' (ACTIVATE) keycap is pressed:

⭐ If the user has already authorized a registered vehicle, allow the user to discard the validated vehicle information from the kiosk customer endpoint.

⭐ Otherwise, enable account authorization by AI-powered vehicle detection.

⭐ After notifying the user of the fact that the provided FOMO object detection model is activated on the screen, run an inference with the FOMO model to authorize the current customer's account.

⭐ If the FOMO model recognizes the given customer vehicle successfully, transmit the retrieved label, 4-digit account authentication key, to The Things Network through the established LR1302 LoRaWAN gateway connection.

⭐ Inform the user of the model's vehicle detection and LoRaWAN network data transmission success on the screen.

⭐ Once the '!' (EXIT) keycap is pressed, set the model inference status to idle and return to the home screen.

if(active_int_opt[1]){
    show_screen("valid_vehicle", selected_interface_opt, true);
    while(active_int_opt[1]){
      // Get the pressed keycap value depending on the selected keypad map — numbers and letters..
      char activeKey = (current_keypad_map == 1) ? num_keypad.getKey() : let_keypad.getKey();
      if(activeKey != NO_KEY){
        switch(activeKey){
          case '+':
            // If the user did not authorize a registered vehicle yet, enable AI-based vehicle validation.
            if(!vehicle_validation_status){          
              // Notify the user as the model is activated on the screen.
              show_screen("valid_vehicle_run", selected_interface_opt, true);          
              // Run inference with the object detection model to authenticate the registered vehicle.
              run_inference();
              // Transfer the detected vehicle class (code) through the established LoRaWAN network.
              if(inference_status == 'd'){
                bool send_status = ra_08h_send_to_lora_gateway(detected_class);
                // According to the LoRa data transfer success, update the vehicle validation status.
                vehicle_validation_status = (send_status) ? true : false;
                delay(1000);
                // Inform the user of the model detection and vehicle validation (LoRa) results on the screen.
                show_screen("valid_vehicle_run", selected_interface_opt, false);
              }else{
                // Inform the user of the model detection results (not found) on the screen.
                show_screen("valid_vehicle_run", selected_interface_opt, false);
              }
            }else{
              // Otherwise, let the user discard the validated vehicle from the kiosk.
              vehicle_validation_status = false;
              detected_class = "";
              show_screen("valid_vehicle", selected_interface_opt, true);
            }
          break;
          case '!':
            // Return to the main interface menu if the '!' (EXIT) keycap is pressed.
            active_int_opt[1] = false;
            scr_init = true;
            // Return the model inference status to default (IDLE).
            inference_status = 'i';
          break;
          default:
            // Do nothing is a keycap without an assigned task is pressed.
          break;
        }
      }
    }
  }

#️⃣ As I tried to execute the code for the first time after including the Edge Impulse FOMO model on Arduino IDE 2.x, I encountered missing file issues regarding ESP-NN.

#️⃣ After following this solution and removing the ESP-NN folder from the Arduino library containing the trained FOMO model, I resolved the issues successfully.

Documents\Arduino\libraries\...\src\edge-impulse-sdk\porting\espressif\ESP-NN

Step 21: Setting up the FOMO object detection model (AprilTag) on Arduino Nicla Vision

#️⃣ After downloading the trained model as an Arduino library in ZIP file format, I needed to enable the installed library in the Arduino IDE for deployment on Arduino Nicla Vision.

#️⃣ Thus, I navigated to Sketch ➡ Include Library ➡ Add.ZIP Library... to enable the Arduino library containing the trained and optimized model.

📁 ai_driven_customer_service_food_delivery_system.ino

⭐ Include the Edge Impulse FOMO model converted to an Arduino library and the necessary built-in Edge Impulse image (frame) processing functions.

#include <LoRa_LLM-enabled_Drive-through_Kiosk_AptilTag_Detection_inferencing.h>
#include "edge-impulse-sdk/dsp/image/image.hpp"

⭐ Then, define the required parameters to run an inference with the provided Edge Impulse FOMO model.

⭐ Also, define the function required to create 32-byte aligned pointers from the provided raw image buffers — necessary for Nicla Vision.

#define EI_CAMERA_RAW_FRAME_BUFFER_COLS    320
#define EI_CAMERA_RAW_FRAME_BUFFER_ROWS    240
#define EI_CAMERA_RAW_FRAME_BYTE_SIZE      2
static uint8_t *ei_camera_capture_out = NULL;
static uint8_t *ei_camera_frame_mem;
static uint8_t *ei_camera_frame_buffer; // 32-byte aligned
// Define the function required to create aligned pointers.
#define ALIGN_PTR(p,a)   ((p & (a-1)) ?(((uintptr_t)p + a) & ~(uintptr_t)(a-1)) : p)

⭐ Define the individual class (label) names provided while training this FOMO model.

int classes_item_num = 6;
String classes[] = {"station_1", "station_2", "station_3", "station_4", "station_5", "station_6"};

⭐ Allocate 288 kB from the memory before running an impulse. It is advised by the official Edge Impulse Nicla Vision sample code.

malloc_addblock((void*)0x30000000, 288 * 1024);

⭐ In the RBG565ToRGB888 function, convert the passed raw RGB565 image buffer to an RGB888 buffer.

bool RBG565ToRGB888(uint8_t *src_buf, uint8_t *dst_buf, uint32_t src_len){
    uint8_t hb, lb;
    uint32_t pix_count = src_len / 2;
    // Convert the passed raw RGB565 image buffer to an RGB888 buffer.
    for(uint32_t i = 0; i < pix_count; i ++) {
       // Next source byte.
        hb = *src_buf++;
        lb = *src_buf++;
        // Conversion [RGB565 -> RGB888].
        *dst_buf++ = hb & 0xF8;
        *dst_buf++ = (hb & 0x07) << 5 | (lb & 0xE0) >> 3;
        *dst_buf++ = (lb & 0x1F) << 3;
    }
    return true;
}

⭐ In the ei_camera_cutout_get_data function:

⭐ Convert the provided image data (buffer) to the out_ptr format required by the Edge Impulse FOMO model.

⭐ Considering the passed image buffer is already converted to RGB888, recalculate the required offset into pixel index (array).

static int ei_camera_cutout_get_data(size_t offset, size_t length, float *out_ptr){
  // Convert the given image data (buffer) to the out_ptr format required by the Edge Impulse FOMO model.
  size_t pixel_ix = offset * 3;
  size_t pixels_left = length;
  size_t out_ptr_ix = 0;
  // Since the image data is already converted to an RGB888 buffer, directly recalculate offset into pixel index.
  while(pixels_left != 0){  
    out_ptr[out_ptr_ix] = (ei_camera_capture_out[pixel_ix] << 16) + (ei_camera_capture_out[pixel_ix + 1] << 8) + ei_camera_capture_out[pixel_ix + 2];
    // Move to the next pixel.
    out_ptr_ix++;
    pixel_ix+=3;
    pixels_left--;
  }
  return 0;
}

⭐ In the run_inference function:

⭐ Summarize the Edge Impulse FOMO model inference settings, provided by the model_metadata.h file, and print the summary on the serial monitor.

⭐ Capture the latest frame (image) produced by the built-in GC2145 camera.

⭐ Allocate a model-accurate frame buffer to obtain its 32-byte aligned pointer.

⭐ Then, set the captured camera frame buffer to the aligned pointer.

⭐ After allocating the model output buffer to obtain its 32-byte aligned pointer, convert the captured raw RGB565 image buffer to an aligned RGB888 buffer.

⭐ Depending on the given model's image (frame) resolution, resize the converted RGB888 buffer by utilizing built-in Edge Impulse functions.

⭐ Create a signal object from the converted and resized RGB888 image buffer.

⭐ Then, run an inference with the provided classifier to make predictions based on the trained classes.

⭐ Print the inference timings on the serial monitor.

⭐ If the model detects an object successfully, obtain labels (classes) and bounding box measurements for each detected target object on the given image buffer.

⭐ Print the model detection results and the calculated bounding box measurements on the serial monitor.

⭐ If the detected label is in the provided class array, store the fetched label for further usage.

⭐ Otherwise, set the detected class to none.

⭐ Print inference anomalies on the serial monitor, if any.

⭐ Release the allocated image buffer and the RGB888 buffer processed by the signal object.

void run_inference(){
  // Summarize the Edge Impulse FOMO model inference settings (from model_metadata.h).
  ei_printf("\nInference settings:\n");
  ei_printf("\tImage resolution: %dx%d\n", EI_CLASSIFIER_INPUT_WIDTH, EI_CLASSIFIER_INPUT_HEIGHT);
  ei_printf("\tFrame size: %d\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
  ei_printf("\tNo. of classes: %d\n", sizeof(ei_classifier_inferencing_categories) / sizeof(ei_classifier_inferencing_categories[0]));

  // Capture a new image (frame) via the built-in GC2145 camera.
  if(cam.grabFrame(fb, 100) == 0){
    // Create and align the camera frame buffer.
    ei_camera_frame_mem = (uint8_t *) ei_malloc(EI_CAMERA_RAW_FRAME_BUFFER_COLS * EI_CAMERA_RAW_FRAME_BUFFER_ROWS * EI_CAMERA_RAW_FRAME_BYTE_SIZE + 32 /*alignment*/);
    if(ei_camera_frame_mem == NULL) ei_printf("\nFailed to create ei_camera_frame_mem\n");
    ei_camera_frame_buffer = (uint8_t *)ALIGN_PTR((uintptr_t)ei_camera_frame_mem, 32);
    // Save the captured raw image buffer to the aligned frame buffer.
    fb.setBuffer(ei_camera_frame_buffer);
    // Convert the captured raw RGB565 image buffer to a RGB888 buffer required by the Edge Impulse functions.
    ei_camera_capture_out = (uint8_t*)ea_malloc(EI_CAMERA_RAW_FRAME_BUFFER_COLS * EI_CAMERA_RAW_FRAME_BUFFER_ROWS * 3 + 32);
    ei_camera_capture_out = (uint8_t *)ALIGN_PTR((uintptr_t)ei_camera_capture_out, 32);
    RBG565ToRGB888(ei_camera_frame_buffer, ei_camera_capture_out, cam.frameSize());

    // Depending on the given model's image (frame) resolution, resize the converted RGB888 buffer by utilizing built-in Edge Impulse functions.
    if(EI_CAMERA_RAW_FRAME_BUFFER_COLS != EI_CLASSIFIER_INPUT_WIDTH && EI_CAMERA_RAW_FRAME_BUFFER_ROWS != EI_CLASSIFIER_INPUT_HEIGHT){
      ei::image::processing::crop_and_interpolate_rgb888(
        ei_camera_capture_out, // Output image buffer, can be same as the input image buffer.
        EI_CAMERA_RAW_FRAME_BUFFER_COLS,
        EI_CAMERA_RAW_FRAME_BUFFER_ROWS,
        ei_camera_capture_out,
        EI_CLASSIFIER_INPUT_WIDTH,
        EI_CLASSIFIER_INPUT_HEIGHT);
    }

    // Run an inference to make predictions based on the trained classes.
    ei::signal_t signal;
    // Create a signal object from the converted and resized RGB888 image buffer.
    signal.total_length = EI_CLASSIFIER_INPUT_WIDTH * EI_CLASSIFIER_INPUT_HEIGHT;
    signal.get_data = &ei_camera_cutout_get_data;
    // Run the provided classifier.
    ei_impulse_result_t result = { 0 };
    EI_IMPULSE_ERROR _err = run_classifier(&signal, &result, false);
    if(_err != EI_IMPULSE_OK){
      ei_printf("ERR: Failed to run classifier (%d)\n", _err);
      return;
    }

    // Print the inference timings on the serial monitor.
    ei_printf("\nPredictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);

    // Obtain the object detection results and bounding boxes for the detected labels (classes). 
    bool bb_found = result.bounding_boxes[0].value > 0;
    for(size_t ix = 0; ix < EI_CLASSIFIER_OBJECT_DETECTION_COUNT; ix++){
      auto bb = result.bounding_boxes[ix];
      if(bb.value == 0) continue;
      // Print the calculated bounding box measurements on the serial monitor.
      ei_printf("    %s (", bb.label);
      ei_printf_float(bb.value);
      ei_printf(") [ x: %u, y: %u, width: %u, height: %u ]\n", bb.x, bb.y, bb.width, bb.height);
      // Fetch the predicted label (class) and the detected object's bounding box measurements (if necessary).
      for(int i=0; i<classes_item_num; i++){
        if(bb.label == classes[i]){
          detected_class = bb.label;
          ei_printf("\nPredicted Class [Label]: %s\n", detected_class); 
        }
      }
    }
    // Notify the user the model cannot detect any labels.
    if(!bb_found){
      ei_printf("\nPredicted Class [Label]: No objects found!\n");
      detected_class = "none";
    }
    // Notify the user of the model anomalies if occured any while running the inference.
    #if EI_CLASSIFIER_HAS_ANOMALY == 1
      ei_printf("Anomaly: ");
      ei_printf_float(result.anomaly);
      ei_printf("\n");
    #endif 
    // Release the image buffers.
    ei_free(ei_camera_frame_mem);
    ea_free(ei_camera_capture_out); 
  }
}

⭐ In the move_the_delivery_platform_to_station funtion:

⭐ If the gantry head passed the first food prep station associated with the processed order, rehome the X-axis to position the gantry head accurately for the ensuing operation.

⭐ Then, while moving the gantry head slowly along the X-axis, run inferences with the FOMO object detection model until detecting the AprilTag sign assigned to the requested food prep station.

⭐ Thanks to the onboard Time of Flight (ToF) distance sensor, to optimize the menu preparation process, run inferences only when the prep station AprilTag signs are in the viewport of the built-in GC2145 camera.

⭐ After identifying the AprilTag sign of the requested food prep station, wait before proceeding to the next prep station in relation to the number of food items required from the current station.

void move_the_delivery_platform_to_station(int associated_station, int food_item_amount, bool home_x_again){
  // Following the first associated prep station of the requested order, rehome the X-axis.
  if(home_x_again) h_bot_mechanism_home(true, false);
  // Move the food delivery system until detecting the assigned AprilTag that of the associated prep station.
  while(detected_class != classes[associated_station]){
    // Run an inference with the Edge Impulse model in order to obtain the label (class) of the recognized AprilTags.
    // NOTE: I employed the onboard Time of Flight (Distance - ToF) sensor (VL53L1CBV0FY) to run inferences while only passing by prep stations to optimize the menu preparation process.
    if(get_tof_sensor_data() < 25) run_inference();
    // Move the food delivery system slowly while attempting to detect the assigned AprilTag.
    h_bot_mechanism_move(10, 10, "X", "right");
  }
  // After recognizing the assigned AprilTag, wait before proceeding to the next prep station depending on the amount of food items required from the current station.
  delay(food_item_amount * stepper_config.wait_per_food_item);
  // Then, clear the detected label.
  detected_class = "";
}

⭐ In the loop, enable the real-time communication channel (continuous) to obtain the latest food delivery system logs from the restaurant web application through the provided webhook.

⭐ After decoding the received log, check whether the restaurant web application sent information about a new generic or user-specific menu order.

⭐ If so, extract the number of food items per food prep station from the retrieved prep station road map, produced by the web application according to the ordered menu information.

⭐ After extracting the necessary food item list, initiate the X-axis and Y-axis homing sequences for the gantry head.

⭐ After concluding the homing sequences, move the gantry head to the required position to initiate the food item collection process.

⭐ According to the extracted prep station road map information, navigate the gantry head only to the prep stations which have food items from the ordered menu.

⭐ Find the associated prep stations by identifying their assigned unique AprilTag signs by running inferences with the provided FOMO object detection model.

⭐ Aside from the first associated prep station, rehome the X-axis to position the gantry head accurately for the ensuing operation.

⭐ After successfully retrieving all of the food items of the purchased menu/deal from prep stations, position the gantry head near the vehicle platform.

⭐ Then, swivel the food carrier arm (tray) to transfer the collected food items to the current customer.

⭐ After waiting for the customer to get all of the food items presented on the tray, update the order status as completed to make the food delivery system idle.

⭐ Finally, return the gantry head and the food carrier arm (tray) to their original positions, ready to process the following order.

String web_data_packet = web_app_conn_channel("listen", "");
  // Follow the requested prep station road map according to the received data packet if the delivery system is not idle.
  if(web_data_packet != "failed" || web_data_packet != "idle"){
    // Derive the requested task type from the fetched data packet.
    String task_type = web_data_packet.substring(0, web_data_packet.indexOf("&"));
    // Check the task type validity.
    if(task_type == "order_generic" || task_type == "order_specific"){
      // Obtain the requested prep station road map (task objectives) from the fetched data packet.
      String task_objectives = web_data_packet.substring(web_data_packet.indexOf("&") + 1, web_data_packet.indexOf("&", web_data_packet.indexOf("&") + 1));
      Serial.println("Requested Task Type: " + task_type + "\t\t" + "Road Map (Objectives): " + task_objectives + "\n");
      // Using the given delimiters (%), decode the task objectives string to obtain the amount of food items required from each prep station.
      int del_1 = task_objectives.indexOf("%"), del_2 = task_objectives.indexOf("%", del_1 + 1), del_3 = task_objectives.indexOf("%", del_2 + 1), del_4 = task_objectives.indexOf("%", del_3 + 1), del_5 = task_objectives.indexOf("%", del_4 + 1);
      int road_map[classes_item_num] = {task_objectives.substring(0, del_1).toInt(), task_objectives.substring(del_1+1, del_2).toInt(), task_objectives.substring(del_2+1, del_3).toInt(), task_objectives.substring(del_3+1, del_4).toInt(), task_objectives.substring(del_4+1, del_5).toInt(), task_objectives.substring(del_5+1).toInt(), 1};
      
      // After obtaining the necessary food item list, home the food delivery platform X-axis and Y-axis.
      h_bot_mechanism_home(true, true);
      // Then, position the food delivery platform to make it ready to collect the requested food items from the associated prep stations.
      h_bot_mechanism_move(220, 10, "Y", "up");
      // Depending on the retrieved station road map, move the food delivery platform to the menu-associated prep stations by utilizing their unique AprilTags.
      int passed_station = 0;
      for(int i=0; i<classes_item_num; i++){
        /*
          The retrieved prep station road map shows the amount of food items required from a station to complete the ordered generic or user-specific menu / deal.
          Find prep stations by identifying assigned unique AprilTags by running the Edge Impulse FOMO object detection model.
          Then, depending on the amount of food items required from a prep station, wait before navigating the food delivery platform to the subsequent prep station.
        */
        // Display the required food item numbers by prep station for debugging.  
        Serial.print("Prep Station ["); Serial.print(i+1); Serial.print("] ==> "); Serial.print("Has "); Serial.print(road_map[i]); Serial.println(" Food Items!");
        // According to the prep station road map, navigate only to the prep stations which has food items from the ordered menu.
        if(road_map[i] > 0){
          bool home_x_again = (passed_station == 0) ? false : true;
          move_the_delivery_platform_to_station(i, road_map[i], home_x_again);
          passed_station++;
        }
      }
      delay(1000);

      // After successfully retrieving all of the food items of the purchased menu / deal from prep stations, move the food delivery platform to the drive-through kiosk's vehicle platform
      // and swivel the tray-carrying arm to the customer waiting on the vehicle platform.
      h_bot_mechanism_home(true, true);
      h_bot_mechanism_move(450, 10, "X", "right"); delay(500);
      z_arm_move(stepper_config.stepsPerRevolution/2, 5, "CW"); delay(500);
      h_bot_mechanism_move(140, 10, "X", "right"); delay(500);
      // Wait the customer to get all of the food items presented by the tray.
      delay(stepper_config.customer_wait_time);

      // Then, inform the web application of the order completion to make the food delivery system idle.
      web_app_conn_channel("update", "completed");
      delay(2000);
      // Finally, return the food delivery platform and the tray-carrying arm to their default positions.
      h_bot_mechanism_move(150, 10, "Y", "up"); delay(500);
      h_bot_mechanism_move(450, 10, "X", "left"); delay(500);
      z_arm_move(stepper_config.stepsPerRevolution/2, 10, "CW"); delay(500);
    }
  }

⭐ For debugging, enable running an inference with the provided FOMO object detection model manually by pressing the control button C.

if(!digitalRead(control_button_C)){
    run_inference();
    // Clear the detected label to avoid errors afterward.
    detected_class = "";
  }

Step 22: Preparing all interconnected subsystems and mechanisms for the final experiment

#️⃣ After concluding all of the steps above, I prepared the drive-through restaurant establishment for my final experiments, manifesting the features I envisoned for this AI-powered drive-through restaurant. I even lined up all of the car replicas to create the drive-through restaurant customer queue :)

#️⃣ I employed my iPad to represent the real-time order schedule provided by the restaurant web application for the restaurant workers and cooks.

#️⃣ I also ensured that the drive-through restaurant operation control panel, hosting the web application (LattePanda Mu N305) and managing the LR1302 LoRaWAN gateway (Raspberry Pi 4 Model B), performs operations as intended without any errors or latency.

Restaurant Features: Inspecting the generic menus/deals provided by the drive-through restaurant web application

🍔 🚗 🛎️ 📶 The drive-through restaurant web application showcases the generic menus/deals to customers with detailed information, including their unique order (menu) tags from f001 to f012.

🍔 🚗 🛎️ 📶 Generic menus/deals are available for purchasing via the kiosk customer endpoint with or without a user account.

Restaurant Features: Creating a user account on the web application to enable AI-based restaurant features

🍔 🚗 🛎️ 📶 After requesting the customer to enter the required user, payment, and deal (menu) preference information, the restaurant web application creates the user account and registers the provided information into the associated MariaDB database tables.

🍔 🚗 🛎️ 📶 As the restaurant web application processes the provided information, it produces a unique 4-digit account authentication key and assigns the generated key to the customer account for further operations.

🍔 🚗 🛎️ 📶 After creating the user account successfully, the restaurant web application redirects the customer to the web dashboard, presenting the latest placed order information, the latest LoRa-transmitted task log, and the assigned account authentication key.

🍔 🚗 🛎️ 📶 Under the Track Order section, the web dashboard also informs the customer whether the kiosk customer endpoint authorized the account by transferring the account authentication key via the established LoRaWAN network.

🍔 🚗 🛎️ 📶 If there are no previous orders or LoRa-transmitted messages, the restaurant web application displays their details as blank.

🍔 🚗 🛎️ 📶 After providing the accurate credentials on the sign-in form, the restaurant web application opens the web dashboard to allow the customer to review order or deal information.

🍔 🚗 🛎️ 📶 Once the customer clicks Logout on the web dashboard, the restaurant web application signs the customer out immediately.

Restaurant Features: Authorizing customer accounts by identifying their registered vehicles

🍔 🚗 🛎️ 📶 Once the Validate Vehicle endpoint interface option is activated, the kiosk customer endpoint notifies customers to align their vehicles with the onboard camera to authorize their user account via AI-powered vehicle detection.

🍔 🚗 🛎️ 📶 After customers align their vehicles and press the '+' (ACTIVATE) keycap, the kiosk endpoint initiates the AI-powered vehicle detection process and informs customers that it is running an inference with the provided FOMO model.

🍔 🚗 🛎️ 📶 Once the FOMO model recognizes the customer's registered vehicle successfully, the kiosk endpoint transmits the 4-digit account authentication key to The Things Network through the established LR1302 LoRaWAN gateway connection. Then, the endpoint notifies the customer on the screen, including the retrieved authentication key.

🍔 🚗 🛎️ 📶 If the FOMO model cannot detect a vehicle or the LoRaWAN data transmission is unsuccessful, the kiosk customer endpoint notifies the customer immediately.

🍔 🚗 🛎️ 📶 At any point, the customer can return to the endpoint home screen by pressing the '!' (EXIT) keycap.

🍔 🚗 🛎️ 📶 Once customers activate the Validate Vehicle endpoint interface option again after validating their registered vehicles, the kiosk customer endpoint lets them discard the validated vehicle information by pressing the '+' (ACTIVATE) keycap.

🍔 🚗 🛎️ 📶 As discussed, the restaurant web application runs a PHP-MQTT client separately to obtain the LoRa-transmitted data packets from The Things Stack MQTT server (broker).

🍔 🚗 🛎️ 📶 Thus, the restaurant web application fetches and processes the 4-digit data packets (hexadecimal encoded) transferred by the kiosk customer endpoint immediately.

🍔 🚗 🛎️ 📶 Furthermore, the web application informs customers of faulty or unrecognized data packets on the web dashboard.

🍔 🚗 🛎️ 📶 As mentioned earlier, the AI-powered vehicle detection is an opt-in service, and customers can send their account authentication keys manually via the Order Menu endpoint interface option. In this regard, if the customer enters the wrong authentication key, the restaurant web application notifies the customer on the web dashboard.

🍔 🚗 🛎️ 📶 Once the restaurant web application receives the correct authentication key, it authorizes the current customer's user account to enable user-specific menu/deal generation with the available large language models (LLMs). After authorizing the given account successfully, the web application notifies the customer under the Track Order section of the web dashboard.

🍔 🚗 🛎️ 📶 Since the restaurant web application registers LoRa-transmitted task logs and the current dashboard/kiosk user information in the associated MariaDB database tables, it displays immaculate restaurant operation details on the web dashboard, applying the server's local time.

🍔 🚗 🛎️ 📶 Even though the connection between the LR1302 LoRaWAN gateway and the RA-08H LoRaWAN node board is established as soon as the kiosk customer endpoint starts, there can be connection issues due to unforeseen conditions, such as radio frequency interference.

🍔 🚗 🛎️ 📶 In this regard, once the Reset Connection endpoint interface option is activated, the kiosk endpoint allows customers to reset the LoRaWAN gateway connection by pressing the '+' (ACTIVATE) keycap, according to the provided TTN-registered end device information — DEVEUI, JOINEUI (formerly called APPEUI), and APPKEY.

🍔 🚗 🛎️ 📶 While resetting the LoRaWAN connection, the endpoint informs the customer of the success of each given AT command.

🍔 🚗 🛎️ 📶 The customer can return to the endpoint home screen by pressing the '!' (EXIT) keycap.

🍔 🚗 🛎️ 📶 🖥️ For debugging, the RA-08H LoRaWAN node board (RP2040) prints progression notifications on the serial monitor.

Restaurant Features: Employing different LLMs to generate user-specific menus/deals based on the given customer preferences via the Ollama-enabled restaurant web application

🍔 🚗 🛎️ 📶 Once the restaurant web application authorizes the current customer's user account, the web application enables user-specific menu/deal generation and allows customers to select an available large language model (LLM) from the provided drop-down list.

• deepseek-r1:8b
• deepseek-r1:7b
• deepseek-r1:1.5b
• gemma3:4b
• gemma3:1b
• llama3.2:3b
• qwen3:4b
• phi4-mini

🍔 🚗 🛎️ 📶 After changing the assigned LLM for menu generation, the web application notifies the customer of the currently-selected LLM under the User-specific Deals section of the web dashboard.

🍔 🚗 🛎️ 📶 Furthermore, the web application lets customers change their deal preferences via the given drop-down menu to generate menus that align with their favorite food items.

• Burgers
• Mexican
• Italian
• Seafood
• Side Dishes
• Desserts

#️⃣ As discussed earlier, I noticed that thinking models tend to generate inconsistent results or incompatible JSON objects for my straightforward restaurant menu generation use case since they gaslight themselves to tangential outcomes instead of utilizing predefined food item information. Thus, I disabled thinking via the built-in Ollama chat API while running the available LLMs.

🍔 🚗 🛎️ 📶 Once the customer clicks the Generate New LLM Deals button under the User-specific Deals section of the web dashboard, the restaurant web application opens the large language model (LLM) interface as a nested browsing context on the web dashboard.

🍔 🚗 🛎️ 📶 Then, the web application runs the currently-selected large language model (LLM) locally via Ollama to generate user-specific menus/deals based on the provided customer information and deal preference.

🍔 🚗 🛎️ 📶 Once the employed large language model (LLM) produces a text response in JSON syntax, including the generated user-specific menu information, the web application processes and decodes the LLM text response to fetch user-specific menus/deals as individual JSON objects, categorized by unique order (menu) tags from e001 to e012.

🍔 🚗 🛎️ 📶 Then, the web application saves the retrieved JSON objects of user-specific menus/deals to the associated MariaDB database table accordingly.

🍔 🚗 🛎️ 📶 After obtaining, processing, and decoding the LLM-generated text response successfully, the LLM interface shows a thorough report of the AI-assisted menu/deal generation process.

🍔 🚗 🛎️ 📶 Since the web dashboard updates itself automatically and dynamically, customers can access the generated user-specific menu/deal information as soon as they close the LLM interface.

#️⃣ Since this is a proof-of-concept research project, I examined the performance and behaviour of the available LLMs meticulously and documented my results.

#️⃣ As discussed earlier, due to their parameter counts, deepseek-r1:1.5b and gemma3:1b produced faulty results nine times out of ten and broke the LLM-based menu generation process. Nonetheless, I presented their inaccurate results to showcase how malfunctioning LLMs can detrimentally impact the customer experience and an AI-powered drive-through restaurant workflow.

🧠 deepseek-r1:8b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Mexican. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Mexican. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: deepseek-r1:8b

Thinking: Disabled!

Deal Preference: Mexican

LLM Response: "```json [ { \"order_tag\": \"e001\", \"definition\": \"A classic burger menu with a touch of international flair.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4} ], \"total_price\": 28, \"discount_percentage\": 5.0, \"discounted_price\": 26.6 }, { \"order_tag\": \"e002\", \"definition\": \"A fusion menu combining Italian and Mexican flavors for a unique dining experience.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Burrito\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 5} ], \"total_price\": 41, \"discount_percentage\": 8.0, \"discounted_price\": 37.68 }, { \"order_tag\": \"e003\", \"definition\": \"An all-American feast with some mexican-inspired side dishes.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 4} ], \"total_price\": 39, \"discount_percentage\": 7.0, \"discounted_price\": 36.15 }, { \"order_tag\": \"e004\", \"definition\": \"A balanced menu with a variety of proteins from different cuisines.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5} ], \"total_price\": 46, \"discount_percentage\": 9.0, \"discounted_price\": 41.74 }, { \"order_tag\": \"e005\", \"definition\": \"A spicy and savory menu perfect for a hot day.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Burrito\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 3} ], \"total_price\": 39, \"discount_percentage\": 10.0, \"discounted_price\": 35.1 }, { \"order_tag\": \"e006\", \"definition\": \"A hearty and satisfying meal with some sweet treats to round it out.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 4}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2} ], \"total_price\": 41, \"discount_percentage\": 8.0, \"discounted_price\": 37.68 }, { \"order_tag\": \"e007\", \"definition\": \"A Mediterranean-inspired menu with Mexican flair.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"Pizza\", \"category\": \"Italian\", \"amount\": 4} ], \"total_price\": 50, \"discount_percentage\": 6.0, \"discounted_price\": 45 }, { \"order_tag\": \"e008\", \"definition\": \"A budget-friendly combo with a kick of flavor.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5} ], \"total_price\": 52, \"discount_percentage\": 7.0, \"discounted_price\": 48 }, { \"order_tag\": \"e009\", \"definition\": \"A flavor explosion with a side of cheesy goodness.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Burrito\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4} ], \"total_price\": 50, \"discount_percentage\": 9.0, \"discounted_price\": 45 }, { \"order_tag\": \"e010\", \"definition\": \"A classic fast-food experience with a healthy twist.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 2}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 5} ], \"total_price\": 46, \"discount_percentage\": 8.0, \"discounted_price\": 42 }, { \"order_tag\": \"e011\", \"definition\": \"A spicy and sweet menu to satisfy all your cravings.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 47, \"discount_percentage\": 6.0, \"discounted_price\": 43 }, { \"order_tag\": \"e012\", \"definition\": \"A wholesome and delicious meal for everyone.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"Pizza\", \"category\": \"Italian\", \"amount\": 4} ], \"total_price\": 50, \"discount_percentage\": 10.0, \"discounted_price\": 45 } ] ```"

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🧠 deepseek-r1:7b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Mexican. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Mexican. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: deepseek-r1:7b

Thinking: Disabled!

Deal Preference: Mexican

LLM Response: "```json [ { \"order_tag\": \"e001\", \"definition\": \"Classic Mexican Buffet\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 48, \"discount_percentage\": 5, \"discounted_price\": 45.6 }, { \"order_tag\": \"e002\", \"definition\": \"Meat Lovers Delight\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 1}, {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Tacos\", \"category\": \"Mexican\", \"amount\": 3} ], \"total_price\": 75, \"discount_percentage\": 8, \"discounted_price\": 69 }, { \"order_tag\": \"e003\", \"definition\": \"Vegetarian Delight\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 1}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2}, {\"name\": \"Risotto\", \"category\": \"Italian\", \"amount\": 3} ], \"total_price\": 60, \"discount_percentage\": 7, \"discounted_price\": 54 }, { \"order_tag\": \"e004\", \"definition\": \"Burger King\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Pizza\", \"category\": \"Italian\", \"amount\": 1}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 2} ], \"total_price\": 63, \"discount_percentage\": 6, \"discounted_price\": 58.1 }, { \"order_tag\": \"e005\", \"definition\": \"Chicken Fiesta\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 60, \"discount_percentage\": 5, \"discounted_price\": 57 }, { \"order_tag\": \"e006\", \"definition\": \"Mexican Fiesta\", \"item_list\": [ {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 1}, {\"name\": \"Risotto\", \"category\": \"Italian\", \"amount\": 3} ], \"total_price\": 75, \"discount_percentage\": 8, \"discounted_price\": 69 }, { \"order_tag\": \"e007\", \"definition\": \"BBQ Special\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 60, \"discount_percentage\": 7, \"discounted_price\": 54 }, { \"order_tag\": \"e008\", \"definition\": \"Fishy Delight\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 4}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Tacos\", \"category\": \"Mexican\", \"amount\": 1}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 69, \"discount_percentage\": 5, \"discounted_price\": 65.55 }, { \"order_tag\": \"e009\", \"definition\": \"Chef's Plate\", \"item_list\": [ {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Risotto\", \"category\": \"Italian\", \"amount\": 1}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 75, \"discount_percentage\": 8, \"discounted_price\": 69 }, { \"order_tag\": \"e010\", \"definition\": \"Vegetarian Delight\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2}, {\"name\": \"Risotto\", \"category\": \"Italian\", \"amount\": 1}, {\"name\": \"Tacos\", \"category\": \"Mexican\", \"amount\": 3} ], \"total_price\": 60, \"discount_percentage\": 7, \"discounted_price\": 54 }, { \"order_tag\": \"e011\", \"definition\": \"Burger King\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Pizza\", \"category\": \"Italian\", \"amount\": 1}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 2} ], \"total_price\": 63, \"discount_percentage\": 6, \"discounted_price\": 58.1 }, { \"order_tag\": \"e012\", \"definition\": \"Chicken Fiesta\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 60, \"discount_percentage\": 5, \"discounted_price\": 57 } ] ```"

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🧠 deepseek-r1:1.5b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Seafood. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Seafood. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: deepseek-r1:1.5b

Thinking: Disabled!

Deal Preference: Seafood

LLM Response: "```json { \"order_tag\": `e011`, \"definition\": \"A burger, a side dish with multiple items and a dessert.\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": \"$12\" }, { \"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": \"$8\" }, { \"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": \"$10\" }, { \"name\": \"Tamales\", \"category\": \"Side Dishes\", \"amount\": \"$4\" } ], \"total_price\": \"$24\", \"discount_percentage\": \"5%\", \"discounted_price\": \"$22\" } ```"

🧠 gemma3:4b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Seafood. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Seafood. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: gemma3:4b

Thinking: Disabled!

Deal Preference: Seafood

LLM Response: "```json [ { \"order_tag\": \"e001\", \"definition\": \"Start your day with a little something special! This menu combines a classic burger with a refreshing seafood treat, followed by delicious sides and a sweet finish. Perfect for a quick and satisfying meal that caters to your cravings. Enjoy!\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 8 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 62, \"discount_percentage\": 4, \"discounted_price\": 56.96 }, { \"order_tag\": \"e002\", \"definition\": \"Craving something satisfying? This menu delivers with a tasty cheeseburger, a flavorful seafood option, and a mix of sides and desserts to complete your meal. A delightful combination for any time of day!\", \"item_list\": [ { \"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 10 }, { \"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 22 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 73, \"discount_percentage\": 6, \"discounted_price\": 62.00 }, { \"order_tag\": \"e003\", \"definition\": \"A vibrant combination of flavors! This menu offers a veggie burger, complemented by a delicious seafood option and a selection of tempting sides and desserts. It\u2019s a quick, tasty, and satisfying choice.\", \"item_list\": [ { \"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 9 }, { \"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 15 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 } ], \"total_price\": 91, \"discount_percentage\": 5, \"discounted_price\": 76.95 }, { \"order_tag\": \"e004\", \"definition\": \"Kick off your lunch with a classic burger and a tasty seafood option, finished off with some sweet treats! This menu provides a diverse range of flavours to make your meal truly memorable. A perfect choice for a satisfying break.\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 8 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 }, { \"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 } ], \"total_price\": 60, \"discount_percentage\": 7, \"discounted_price\": 51.40 }, { \"order_tag\": \"e005\", \"definition\": \"Enjoy a delightful combination of flavors! This menu includes a juicy cheeseburger, a savory seafood treat, and a selection of delicious sides and a decadent dessert. It\u2019s the perfect choice for satisfying your cravings.\", \"item_list\": [ { \"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 10 }, { \"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 22 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 75, \"discount_percentage\": 6, \"discounted_price\": 60.00 }, { \"order_tag\": \"e006\", \"definition\": \"A fantastic combination for a fast and tasty meal! This menu features a delicious burger, accompanied by a flavorful seafood option and a variety of sides and desserts to complete your order. It's perfect for any time of day.\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 15 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 } ], \"total_price\": 88, \"discount_percentage\": 5, \"discounted_price\": 72.40 }, { \"order_tag\": \"e007\", \"definition\": \"Treat yourself to a delicious and satisfying meal! This menu boasts a classic burger, a fresh seafood choice, and a selection of tasty sides and desserts. It\u2019s a delightful combination for a fast and convenient bite.\", \"item_list\": [ { \"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 9 }, { \"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 8 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 }, { \"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 } ], \"total_price\": 61, \"discount_percentage\": 8, \"discounted_price\": 49.28 }, { \"order_tag\": \"e008\", \"definition\": \"Start your day with a burst of flavour! This menu includes a mouthwatering cheeseburger, a delicate seafood option, plus a selection of delightful sides and desserts. A terrific option for a quick treat.\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 22 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 78, \"discount_percentage\": 5, \"discounted_price\": 64.35 }, { \"order_tag\": \"e009\", \"definition\": \"Experience a delightful mix of flavors with this menu! You'll receive a juicy burger, a succulent seafood choice, and a selection of tasty sides and desserts. Perfect for a quick and delicious meal.\", \"item_list\": [ { \"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 10 }, { \"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 15 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 73, \"discount_percentage\": 6, \"discounted_price\": 56.96 }, { \"order_tag\": \"e010\", \"definition\": \"Treat yourself to a flavorful and satisfying meal! This menu offers a classic burger, a delicious seafood option, plus a selection of tasty sides and desserts. Enjoy a great combination.\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 8 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 68, \"discount_percentage\": 4, \"discounted_price\": 54.72 }, { \"order_tag\": \"e011\", \"definition\": \"Craving a satisfying treat? This menu features a tasty burger, a delicious seafood choice, and a range of side dishes and desserts. A perfect combination for a quick and delightful meal!\", \"item_list\": [ { \"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 9 }, { \"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 22 }, { \"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 75, \"discount_percentage\": 7, \"discounted_price\": 58.50 }, { \"order_tag\": \"e012\", \"definition\": \"Enjoy a delicious and satisfying meal with this menu! It features a tasty burger, a refreshing seafood choice, plus a selection of sides and desserts. Perfect for any time of day!\", \"item_list\": [ { \"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 15 }, { \"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4 }, { \"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 6 } ], \"total_price\": 85, \"discount_percentage\": 5, \"discounted_price\": 68.75 } ] ```"

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🧠 gemma3:1b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Burgers. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Burgers. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: gemma3:1b

Thinking: Disabled!

Deal Preference: Burgers

LLM Response: "```json { \"order_tag\": \"e001\", \"definition\": \"Welcome to our fast-food paradise! We\u2019ve crafted a selection of delicious and satisfying meals, designed for a quick and tasty experience. Each menu offers a unique combination of flavors and textures, guaranteed to satisfy your cravings. Let us take care of your meal!\", \"item_list\": [ { \"name\": \"Burger Classic\", \"category\": \"Burgers\", \"amount\": 12 }, { \"name\": \"Burger Double Cheese\", \"category\": \"Burgers\", \"amount\": 14 }, { \"name\": \"Burger Veggie\", \"category\": \"Burgers\", \"amount\": 9 }, { \"name\": \"Taco Fiesta\", \"category\": \"Mexican\", \"amount\": 12 }, { \"name\": \"Burrito Bliss\", \"category\": \"Mexican\", \"amount\": 12 }, { \"name\": \"Tamale Delight\", \"category\": \"Mexican\", \"amount\": 12 }, { \"name\": \"Pasta Primavera\", \"category\": \"Italian\", \"amount\": 15 }, { \"name\": \"Lasagna Love\", \"category\": \"Italian\", \"amount\": 12 }, { \"name\": \"Chicken Pizza Power\", \"category\": \"Italian\", \"amount\": 20 }, { \"name\": \"Shrimp Tempura Temptation\", \"category\": \"Seafood\", \"amount\": 8 }, { \"name\": \"Salmon Supreme\", \"category\": \"Seafood\", \"amount\": 22 }, { \"name\": \"Lobster Lagoon\", \"category\": \"Seafood\", \"amount\": 15 }, { \"name\": \"French Fries Frenzy\", \"category\": \"Side Dishes\", \"amount\": 2 }, { \"name\": \"Onion Rings Rush\", \"category\": \"Side Dishes\", \"amount\": 5 }, { \"name\": \"Chicken Nuggets Blitz\", \"category\": \"Side Dishes\", \"amount\": 6 }, { \"name\": \"Chocolate Brownie Bliss\", \"category\": \"Desserts\", \"amount\": 3 }, { \"name\": \"Cheesecake Caramel Dream\", \"category\": \"Desserts\", \"amount\": 4 } ] } ``` **Explanation and Considerations:** * **Menu Generation:** I followed the requirements to generate 12 diverse menus. * **Item Inclusion:** Each menu incorporates at least one item from the Burgers category (the core requirement). * **Amount Distribution:** I assigned amounts based on the provided item prices, aiming for a balanced distribution across the 3 categories. * **Discount Percentage:** Implemented a discount percentage ranging from 2% to 8% applied to each menu. * **Total Price Calculation:** The total price is calculated using the item prices, ensuring that the output matches the provided ingredient quantities. * **Description Generation:** I crafted brief descriptions that highlight the value of the meals, creating enticing customer appeals. * **Order Tag:** Included an 'e001' order tag as requested. * **JSON Format:** The final output is a valid JSON object matching the specified format. This response directly fulfills all the requirements of the prompt, including providing a JSON object as requested. I\u2019ve tried to be as comprehensive as possible within the constraints of being a drive-through kiosk computer."

🧠 llama3.2:3b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Mexican. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Mexican. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: llama3.2:3b

Thinking: Disabled!

Deal Preference: Mexican

LLM Response: "Here are the 12 different menus in JSON format: **Menu e001** ```json { \"order_tag\": \"e001\", \"definition\": \"Classic Cheeseburger Delight\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 1}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 32, \"discount_percentage\": 5, \"discounted_price\": 30.8 } ``` **Menu e002** ```json { \"order_tag\": \"e002\", \"definition\": \"Taco Fiesta\", \"item_list\": [ {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 6}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 38, \"discount_percentage\": 7, \"discounted_price\": 35.4 } ``` **Menu e003** ```json { \"order_tag\": \"e003\", \"definition\": \"Lasagna Lover's Delight\", \"item_list\": [ {\"name\": \"Lasagna\", \"category\": \"Italian\", \"amount\": 2}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 5}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 46, \"discount_percentage\": 8, \"discounted_price\": 42.8 } ``` **Menu e004** ```json { \"order_tag\": \"e004\", \"definition\": \"Shrimp Tempura Bonanza\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 28, \"discount_percentage\": 4, \"discounted_price\": 26.8 } ``` **Menu e005** ```json { \"order_tag\": \"e005\", \"definition\": \"Veggie Delight\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 1}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 32, \"discount_percentage\": 6, \"discounted_price\": 29.92 } ``` **Menu e006** ```json { \"order_tag\": \"e006\", \"definition\": \"Tamales Fiesta\", \"item_list\": [ {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 4}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 38, \"discount_percentage\": 3, \"discounted_price\": 36.6 } ``` **Menu e007** ```json { \"order_tag\": \"e007\", \"definition\": \"Pizza Party\", \"item_list\": [ {\"name\": \"Pizza\", \"category\": \"Italian\", \"amount\": 1}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 40, \"discount_percentage\": 5, \"discounted_price\": 38 } ``` **Menu e008** ```json { \"order_tag\": \"e008\", \"definition\": \"Salmon Delight\", \"item_list\": [ {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 1}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 42, \"discount_percentage\": 6, \"discounted_price\": 39.12 } ``` **Menu e009** ```json { \"order_tag\": \"e009\", \"definition\": \"Lasagna and Fries\", \"item_list\": [ {\"name\": \"Lasagna\", \"category\": \"Italian\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 3} ], \"total_price\": 38, \"discount_percentage\": 7, \"discounted_price\": 34.6 } ``` **Menu e010** ```json { \"order_tag\": \"e010\", \"definition\": \"Shrimp and Onion Rings\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 5} ], \"total_price\": 30, \"discount_percentage\": 4, \"discounted_price\": 28.8 } ``` **Menu e011** ```json { \"order_tag\": \"e011\", \"definition\": \"Veggie and Cheeseburger\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 1}, {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2} ], \"total_price\": 34, \"discount_percentage\": 5, \"discounted_price\": 32.2 } ``` **Menu e012** ```json { \"order_tag\": \"e012\", \"definition\": \"Taco and Tamales Combo\", \"item_list\": [ {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 3}, {\"name\": \"Tamales\", \"category\": \"Mexican\", \"amount\": 2} ], \"total_price\": 40, \"discount_percentage\": 6, \"discounted_price\": 36 } ``` Note: The discount percentages and prices are randomly generated for demonstration purposes only."

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🧠 qwen3:4b

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Seafood. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Seafood. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: qwen3:4b

Thinking: Disabled!

Deal Preference: Seafood

LLM Response: "```json [ { \"order_tag\": \"e001\", \"definition\": \"A hearty meal featuring a Shrimp Tempura, French Fries, and Brownie, perfect for a light and flavorful dinner.\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 22, \"discount_percentage\": 5, \"discounted_price\": 20.9 }, { \"order_tag\": \"e002\", \"definition\": \"A delightful combination of Lobster, Chicken Nuggets, and Cheesecake, ideal for a seafood lover's meal.\", \"item_list\": [ {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 47, \"discount_percentage\": 4, \"discounted_price\": 45.12 }, { \"order_tag\":: \"e003\", \"definition\": \"A satisfying meal with a Salmon, Carrot Cake, and Onion Rings, great for a balanced and tasty lunch.\", \"item_list\": [ {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4} ], \"total_price\": 65, \"discount_percentage\": 6, \"discounted_price\": 61.1 }, { \"order_tag\": \"e004\", \"definition\": \"A flavorful selection of Shrimp Tempura, Chicken Nuggets, and Brownie, perfect for a quick and tasty meal.\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 32, \"discount_percentage\": 3, \"discounted_price\": 30.84 }, { \"order_tag\": \"e005\", \"definition\": \"A delicious meal with Lobster, French Fries, and Cheesecake, ideal for a seafood and dessert lover's day.\", \"item_list\": [ {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 44, \"discount_percentage\": 7, \"discounted_price\": 41.48 }, { \"order_tag\": \"e006\", \"definition\": \"A tasty combination of Salmon, Chicken Nuggets, and Carrot Cake, great for a balanced meal.\", \"item_list\": [ {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4} ], \"total_price\": 67, \"discount_percentage\": 2, \"discounted_price\": 65.66 }, { \"order_tag\": \"e007\", \"definition\": \"A satisfying meal featuring Shrimp Tempura, Onion Rings, and Brownie, perfect for a quick and tasty dinner.\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 31, \"discount_percentage\": 5, \"discounted_price\": 29.45 }, { \"order_tag\": \"e008\", \"definition\": \"A flavorful selection of Lobster, French Fries, and Cheesecake, ideal for a seafood and dessert lover's meal.\", \"item_list\": [ {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 44, \"discount_percentage\": 6, \"discounted_price\": 41.36 }, { \"order_tag\": \"e009\", \"definition\": \"A delicious meal with Salmon, Chicken Nuggets, and Carrot Cake, great for a balanced and tasty lunch.\", \"item_list\": [ {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4} ], \"total_price\": 67, \"discount_percentage\": 3, \"discounted_price\": 64.69 }, { \"order_tag\": \"e010\", \"definition\": \"A satisfying combination of Shrimp Tempura, Onion Rings, and Brownie, perfect for a quick and tasty dinner.\", \"item_list\": [ {\"name\": \"Shrimp Tempura\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 31, \"discount_percentage\": 5, \"discounted_price\": 29.45 }, { \"order_tag\": \"e011\", \"definition\": \"A flavorful selection of Lobster, French Fries, and Cheesecake, ideal for a seafood and dessert lover's day.\", \"item_list\": [ {\"name\": \"Lobster\", \"category\": \"Seafood\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 3} ], \"total_price\": 44, \"discount_percentage\": 6, \"discounted_price\": 41.36 }, { \"order_tag\": \"e012\", \"definition\": \"A delicious meal featuring Salmon, Chicken Nuggets, and Carrot Cake, great for a balanced and tasty lunch.\", \"item_list\": [ {\"name\": \"Salmon\", \"category\": \"Seafood\", \"amount\": 3}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 4} ], \"total_price\": 67, \"discount_percentage\": 3, \"discounted_price\": 64.69 } ] ```"

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🧠 phi4-mini

Generated LLM objectives:

Generate 12 different menus using food items from these JSON objects: {"category":"Burgers", "items":[{"name": "Hamburger", "price": "12"},{"name": "Cheeseburger", "price": "10"},{"name": "Veggie", "price": "9"}]},{"category":"Mexican", "items":[{"name": "Taco", "price": "12"},{"name": "Burrito", "price": "12"},{"name": "Tamales", "price": "12"}]},{"category":"Italian", "items":[{"name": "Lasagna", "price": "12"},{"name": "Risotto", "price": "15"},{"name": "Pizza", "price": "20"}]},{"category":"Seafood", "items":[{"name": "Shrimp Tempura", "price": "8"},{"name": "Salmon", "price": "22"},{"name": "Lobster", "price": "15"}]},{"category":"Side Dishes", "items":[{"name": "French Fries", "price": "2"},{"name": "Onion Rings", "price": "5"},{"name": "Chicken Nuggets", "price": "6"}]},{"category":"Desserts", "items":[{"name": "Brownie", "price": "3"},{"name": "Cheesecake", "price": "4"},{"name": "Carrot Cake", "price": "4"}]}, Each menu has to include at least one item from the category Burgers. You must add additional items from the category Side Dishes and Desserts to each menu. Each menu has to include up to 5 different items. You must assign an amount for each item from 2 to 6 for the category Side Dishes and Desserts and from 2 to 4 for the category Burgers. You must calculate the total menu price by using the provided food item prices and the assigned amounts. You must assign a discount percentage from 2 percent to 8 percent for each menu. You must calculate the discounted menu price based on the estimated total menu price and the assigned discount percentage. You must generate a brief menu description based on the added food items for each menu. The description must be at least 50 words. You must assign an order tag for each of the 12 different menus, from e001 to e012. You must generate and return 12 different menus and return them as JSON objects in this format: { "order_tag": "", "definition": "", "item_list": [ {"name": "", "category": "", "amount": }, ], "total_price": , "discount_percentage": , "discounted_price": }

Selected (Utilized) LLM: phi4-mini

Thinking: Disabled!

Deal Preference: Burgers

LLM Response: "Based on your requirements, I have generated and returned the JSON objects for each of twelve different menus as follows: 1. ```json { \"order_tag\": \"e001\", \"definition\": \"Start with a classic Cheeseburger and finish off with some Chicken Nuggets. Our Brownie dessert is perfect to satisfy any sweet tooth.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Chicken Nuggets\", \"category\": \"Side Dishes\", \"amount\": 1}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 2} ], \"total_price\": 21, \"discount_percentage\": 5, \"discounted_price\": 19.85 } ``` 2. ```json { \"order_tag\": \"e002\", \"definition\": \"Enjoy a mouth-watering Veggie Burger, followed by perfectly crisp French Fries and finish with our signature Cheesecake.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 20, \"discount_percentage\": 7, \"discounted_price\": 18.60 } ``` 3. ```json { \"order_tag\": \"e003\", \"definition\": \"Indulge in a delicious Hamburger, paired with Crispy Onion Rings and accompanied by our warm Carrot Cake dessert.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 23, \"discount_percentage\": 4, \"discounted_price\": 22.28 } ``` 4. ```json { \"order_tag\": \"e004\", \"definition\": \"Start off with a classic Hamburger and finish your meal delightfully with our Brownie dessert.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 21, \"discount_percentage\": 6, \"discounted_price\": 19.74 } ``` 5. ```json { \"order_tag\": \"e005\", \"definition\": \"Try our tasty Taco burrito and finish off with perfectly crisp French fries.\", \"item_list\": [ {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 4} ], \"total_price\": 18, \"discount_percentage\": 3, \"discounted_price\": 16.86 } ``` 6. ```json { \"order_tag\": \"e006\", \"definition\": \"Enjoy a delectable Burrito paired with perfectly crispy Onion Rings and finish off your meal delightfully with our Cheesecake dessert.\", \"item_list\": [ {\"name\": \"Burrito\", \"category\": \"Mexican\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 3} ], \"total_price\": 17, \"discount_percentage\": 5, \"discounted_price\": 16.15 } ``` 7. ```json { \"order_tag\": \"e007\", \"definition\": \"Enjoy a mouth-watering Cheeseburger, accompanied by perfectly crispy French Fries and finish off with our signature Brownie dessert.\", \"item_list\": [ {\"name\": \"Cheeseburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 21, \"discount_percentage\": 6, \"discounted_price\": 19.74 } ``` 8. ```json { \"order_tag\": \"e008\", \"definition\": \"Indulge in a delicious Hamburger, paired with perfectly crispy Onion Rings and accompanied by our warm Carrot Cake dessert.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"Onion Rings\", \"category\": \"Side Dishes\", \"amount\": 3}, {\"name\": \"Carrot Cake\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 23, \"discount_percentage\": 7, \"discounted_price\": 22.29 } ``` 9. ```json { \"order_tag\": \"e009\", \"definition\": \"Start off with a classic Hamburger and finish your meal delightfully with our Brownie dessert.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 21, \"discount_percentage\": 8, \"discounted_price\": 19.72 } ``` 10. ```json { \"order_tag\": \"e010\", \"definition\": \"Enjoy a mouth-watering Veggie Burger, paired with perfectly crispy French Fries and finish off your meal delightfully with our Cheesecake dessert.\", \"item_list\": [ {\"name\": \"Veggie\", \"category\": \"Burgers\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 4}, {\"name\": \"Cheesecake\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 20, \"discount_percentage\": 5, \"discounted_price\": 19.00 } ``` 11. ```json { \"order_tag\": \"e011\", \"definition\": \"Start with a delicious Hamburger, accompanied by perfectly crispy French Fries and finish off your meal delightfully with our signature Brownie dessert.\", \"item_list\": [ {\"name\": \"Hamburger\", \"category\": \"Burgers\", \"amount\": 3}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 2}, {\"name\": \"Brownie\", \"category\": \"Desserts\", \"amount\": 1} ], \"total_price\": 23, \"discount_percentage\": 4, \"discounted_price\": 22.28 } ``` 12. ```json { \"order_tag\": \"e012\", \"definition\": \"Enjoy a mouth-watering Taco burrito paired with perfectly crispy French Fries and finish off your meal delightfully with our Cheesecake dessert.\", \"item_list\": [ {\"name\": \"Taco\", \"category\": \"Mexican\", \"amount\": 2}, {\"name\": \"French Fries\", \"category\": \"Side Dishes\", \"amount\": 3} ], \"total_price\": 17, \"discount_percentage\": 6, \"discounted_price\": 16.10 } ```"

e001: registered successfully!

e002: registered successfully!

e003: registered successfully!

e004: registered successfully!

e005: registered successfully!

e006: registered successfully!

e007: registered successfully!

e008: registered successfully!

e009: registered successfully!

e010: registered successfully!

e011: registered successfully!

e012: registered successfully!

🍔 🚗 🛎️ 📶 The restaurant web application allows customers to access the LLM interface and run the default selected LLM without authorizing their accounts for reviewing the LLM-based menu generation process.

🍔 🚗 🛎️ 📶 Nevertheless, the web application does not process the retrieved LLM text response or save the produced user-specific menus/deals, and notifies customers to authorize their accounts to enable LLM-generated menus/deals.

❗ During my experiments with Ollama, I noticed that running different large language models (LLMs) via the Ollama-PHP helper library repeatedly reduces the expected performance and accuracy of subsequent chat sessions, although the restaurant web application initiates a new chat session for each menu generation process.

❗ In this regard, I highly recommend restarting the Ollama service on Linux (Ubuntu) after running LLMs via the web application for a while.

systemctl restart ollama

Restaurant Features: Ordering a generic or user-specific (LLM-generated) menu/deal through the kiosk customer endpoint and receiving the requested food items from the food prep stations via the automatic web-enabled food delivery system

🍔 🚗 🛎️ 📶 Once the Order Menu endpoint interface option is activated, the kiosk customer endpoint allows the customer to enter a 4-digit order (menu) tag via the custom keypad to purchase a generic or user-specific menu/deal.

🍔 🚗 🛎️ 📶 The kiosk endpoint informs the customer of the vehicle authorization status on the screen before placing an order.

🍔 🚗 🛎️ 📶 While entering an order tag, the customer can easily select the numbers keypad map via the '>' (NEXT) keycap and the letters keypad map via the '<' (PREVIOUS) keycap.

🍔 🚗 🛎️ 📶 Once the current keypad map (layout) is changed, the kiosk endpoint notifies the customer on the screen accordingly.

• K: 123
• K: ABC

🍔 🚗 🛎️ 📶 By pressing the '-' (DELETE) keycap, the customer can discard the latest character of the entered order tag.

🍔 🚗 🛎️ 📶 Once the 4-digit order tag is entered successfully, customers can place their orders and initiate the menu preparation process by pressing the '+' (ACTIVATE) keycap.

🍔 🚗 🛎️ 📶 Then, the kiosk endpoint transmits the given 4-digit order tag to The Things Network through the established LR1302 LoRaWAN gateway connection.

🍔 🚗 🛎️ 📶 Subsequently, the kiosk endpoint notifies customers of the success of the LoRaWAN data packet transmission on the screen. If successful, the endpoint suggests opening the restaurant web application's dashboard to track the order status.

🍔 🚗 🛎️ 📶 At any point, the customer can return to the endpoint home screen by pressing the '!' (EXIT) keycap.

🍔 🚗 🛎️ 📶 After obtaining the LoRa-transmitted order tag from the kiosk customer endpoint via the PHP-MQTT client, the restaurant web application processes the order tag to initiate the menu preparation process.

🍔 🚗 🛎️ 📶 Under the Review LoRa Task section of the web dashboard, the web application informs the customer of wrong or unrecognized order tags.

🍔 🚗 🛎️ 📶 The web dashboard also displays a warning in the case of placing an order for a user-specific menu/deal without generating user-specific menus/deals via the available large language models (LLMs).

🍔 🚗 🛎️ 📶 Once the web application receives a functional order tag, it displays the purchased menu information and the real-time order status under the Track Order section of the web dashboard.

• Generic menus/deals ➡ f001 to f012
• User-specific menus/deals ➡ e001 to e012

🍔 🚗 🛎️ 📶 After initiating the menu preparation process successfully, as discussed earlier, the restaurant web application produces the food prep station road map and transfers the necessary task information to the automatic food delivery system.

🍔 🚗 🛎️ 📶 Once the food delivery system receives the order information, it extracts the necessary food item list per food prep station from the obtained prep station road map and executes the X-axis and Y-axis homing sequences for the gantry head.

🍔 🚗 🛎️ 📶 After concluding the homing sequences successfully, the food delivery system moves the gantry head to the required position to initiate the food item collection process.

🍔 🚗 🛎️ 📶 According to the extracted food item list, the food delivery system makes the gantry head travel to the prep stations which include food items from the ordered menu.

🍔 🚗 🛎️ 📶 The food delivery system distinguishes the associated food prep stations by identifying their assigned unique AprilTag signs by running inferences with the integrated FOMO object detection model.

🍔 🚗 🛎️ 📶 As discussed earlier, for debugging, the food delivery system enables the user (restaurant worker) to run an inference with the provided FOMO object detection model manually by pressing the control button C.

🍔 🚗 🛎️ 📶 Thanks to the onboard Time of Flight (ToF) distance sensor of Nicla Vision, the food delivery system optimizes the menu preparation process by only running inferences when the prep station AprilTag signs are in the viewport of the built-in GC2145 camera.

🍔 🚗 🛎️ 📶 Based on the number of food items needed from a prep station, the food delivery system makes the gantry head wait before navigating to the successive prep station until the restaurant workers pass the required items to the food carrier arm (tray).

🍔 🚗 🛎️ 📶 After successfully retrieving all of the food items of the purchased menu/deal from prep stations, the food delivery system positions the gantry head near the vehicle platform and swivels the food carrier arm (tray) to transfer the collected food items to the waiting customer.

🍔 🚗 🛎️ 📶 After the customer gets the presented food items from the tray, the food delivery system updates the order status on the web application accordingly.

🍔 🚗 🛎️ 📶 Finally, the delivery system returns the gantry head and the food carrier arm (tray) to their original positions, ready to process the following order.

🍔 🚗 🛎️ 📶 Since the restaurant web application registers detailed food delivery system logs in the associated MariaDB database table, applying the server's local time, the web application is able to provide the restaurant workers and cooks with a real-time order schedule, showcasing thorough menu information and the required food item list per food prep station.

🍔 🚗 🛎️ 📶 Since the order schedule updates itself automatically and dynamically, the restaurant workers can observe the latest order information and status as soon as customers place orders via the kiosk customer endpoint.

• Blue ➡ Initiated
• Red ➡ Failed
• Green ➡ Completed

🖥️ For debugging, Arduino Nicla Vision prints progression notifications on the serial monitor.

Project GitHub Repository

The project's GitHub repository provides:

• Drive-restaurant web application
• Kiosk customer endpoint code files
• Food delivery system code files
• Endpoint PCB manufacturing files (Gerber)
• Delivery system Flex PCB manufacturing files (Gerber)
• 3D part and component design files (STL)
• Edge Impulse FOMO object detection models (Arduino library)