Build your own Portable AI Agent

Build Your Own Portable AI Agent with TuyaOpen

This project demonstrates how to build your own portable AI agent using TuyaOpen, an open-source SDK that enables easy and fast integration of multimodal AI capabilities onto almost any hardware. In this tutorial, we'll use the Waveshare T5AI Touch AMOLED 1.75" development board to create a fully functional AI-powered device that can understand voice commands, have conversations, and display information on a beautiful AMOLED screen.

What is TuyaOpen?

TuyaOpen is a flexible, cross-platform C/C++ SDK developed by Tuya to power next-generation AI agent hardware. It supports a wide range of hardware platforms including Tuya T-Series WIFI/BT MCUs, Raspberry Pi, ESP32 platforms, and more. By connecting seamlessly to Tuya Cloud, TuyaOpen enables devices with low-latency multimodal AI features like speech recognition, vision processing, and sensor-based capabilities.

Key Features of TuyaOpen

With TuyaOpen, you can:

Build smart hardware products featuring speech technologies such as:

• ASR (Automatic Speech Recognition) - Convert speech to text
• KWS (Keyword Spotting) - Detect wake words and commands
• TTS (Text-to-Speech) - Convert text responses to natural speech
• STT (Speech-to-Text) - Real-time speech transcriptio

Integrate with leading AI models and platforms such as:

• ChatGPT
• Gemini
• Qwen
• Deepseek
• Claude
• And many more

Enable cloud connectivity for:

• Remote device control
• Real-time monitoring
• OTA (Over-The-Air) firmware updates
• Device management through Tuya Cloud

Target multiple connectivity options:

• Bluetooth (BLE 5.4)
• Wi-Fi (802.11 b/g/n/ax)
• Ethernet
• And more

Benefit from built-in security:

• Device authentication
• Data encryption
• Secure cloud communication

Whether you're building smart home products, industrial IoT solutions, or custom AI-powered devices, TuyaOpen provides the tools and easy ways to help you rapidly develop and deploy your ideas across multiple platforms.

Hardware Introduction

Waveshare T5AI Touch AMOLED 1.75" Development Board

The Waveshare T5AI Touch AMOLED 1.75" is a high-performance, highly integrated microcontroller development board designed by Waveshare. In a compact form factor, it features a stunning 1.75-inch capacitive HD AMOLED screen, RTC (Real-Time Clock), a 6-axis IMU (Inertial Measurement Unit), audio codecs, and other peripherals on board for easy development and integration into products.

Key Hardware Specifications

Main Processor:

• Armv8-M STAR-MC1 MCU running at up to 480 MHz
• 640KB SRAM for fast data processing
• 64KB ROM for bootloader and system code
• 16MB PSRAM for large data buffers
• 8MB external Flash for application storage

Connectivity:

• 2.4GHz Wi-Fi (802.11 b/g/n/ax) with onboard antenna
• Bluetooth 5.4 (Bluetooth LE) with onboard antenna
• USB Type-C interface for programming and power

Display:

• 1.75" capacitive touch HD AMOLED screen
• Resolution: 466×466 pixels
• 16.7M colors for vibrant visuals
• CO5300 driver chip (AMOLED controller)
• CST9217 capacitive touch chip
• Uses QSPI and I2C to save IO pins
• AMOLED advantages: High contrast, wide viewing angles, rich colors, fast response, thin design, low power consumption

Sensors and Peripherals:

• QMI8658 6-axis IMU (3-axis accelerometer + 3-axis gyroscope)
• PCF85063 RTC chip for precise timekeeping and leap year handling
• 2 audio ADCs for microphone input
• 1 audio DAC for speaker output
• Programmable PWR and KEY side buttons for custom development

Power and Expansion:

• 3.7V MX1.25 lithium battery recharge/discharge header
• 8PIN 2.54mm header: 5 GPIOs and 5 IO pads for peripherals/debugging
• TF card slot for extra storage and quick data transfer

More details about this kit:Waveshare T5-E1-Touch-AMOLED-1.75 Wiki

About the Main Wi-Fi/BT MCU: Tuya T5-E1

The Tuya T5-E1 is a custom ARM MCU built specifically for device-cloud integration applications. It's a powerful chip designed to be a Swiss Army knife for building device-cloud AI agents and multimodal hardware.

This makes the T5-E1 an ideal choice for AI agent applications that require both local processing power and cloud connectivity.

Prerequisites

Before you begin, make sure you have the following:

Hardware Requirements

• ✅ Waveshare T5AI Touch AMOLED 1.75" development board
• ✅ USB Type-C cable for programming and power
• ✅ Computer (Windows, Linux, or macOS)

Accounts and Services

• ✅ Tuya Developer Account (free) - Sign up here
• ✅ TuyaOpen Access Key Licence (see Device Authorization section)

Development Environment Setup

You can use Windows, macOS, or Linux as your development environment — all are fully supported and effective for developing with the T5AI board and TuyaOpen SDK.

📦 TuyaOpen Environment Setup

For step-by-step environment setup instructions—including installing Python, Git, and required dependencies—please refer to the official TuyaOpen Quick Start guide:

See the 👉 Environment Setup Guide (TuyaOpen Docs)

This guide will walk you through all the required tooling, recommended best practices (like using virtual environments), and verification steps specific to your system (Windows, macOS, or Linux).

Once your environment is ready, return here to continue with firmware building and application development.

Building Your Firmware

This section will guide you through the process of building your own firmware for the AI agent.

Step 1: Navigate to the Application Directory

cd apps/tuya.ai/your_chat_bot

The your_chat_bot application is a complete AI chatbot example that demonstrates:

• Voice input through microphone
• Speech recognition (ASR)
• AI conversation with LLMs
• Text-to-speech (TTS) output
• Real-time chat display on screen

Step 2: Select Your Development Board

Use the tos config choice command to select the Waveshare T5AI AMOLED 1.75" board:

tos.py config choice

Step 3: Configure Build Options (Optional)

If you need to customize the build configuration, use the menuconfig tool:

tos.py config menu

This opens an interactive configuration menu where you can:

• Enable/disable features
• Change localization language
• Configure audio settings
• Adjust display settings
• Configure network options
• And much more...

After making changes, save the configuration:

tos.py config save

This command saves your configuration settings in a temporary app_default.config cache, which will be used for future builds.

Step 4: Build the Firmware

Now compile the project:

tos.py build

The build process will:

• Configure the build system
• Compile all source files
• Link the firmware
• Generate the binary file

Note: The first build may take several minutes as it compiles all dependencies. Subsequent builds will be much faster.

Expected output: 📷

The compiled firmware will be located in the {project dir}/.build/bin directory.

The ***_QIO_***.bin file is the full image firmware—use this file to flash your entire device.

Flashing the Firmware

Once your firmware is built, you need to flash it to your development board. There are two methods: CLI (Command Line) and GUI Tool.

Method 1: CLI Flashing (Recommended)

The easiest way is to use the one-click flash command:

tos.py flash

This command will:

• Detect the connected device
• Erase the flash memory
• Write the new firmware
• Verify the flash

or you can specify the serial port manually:

Linux/macOS:

tos.py flash --port /dev/ttyUSB0

Windows:

tos.py flash --port COM3

To find your serial port:

• Linux:ls /dev/ttyUSB* or ls /dev/ttyACM*
• macOS:ls /dev/cu.*
• Windows: Check Device Manager under "Ports (COM & LPT)"

Quick Tip: The T5E1 board features two serial COM ports:

• Channel A: Used for flashing firmware (baud rate dynamic, any baud rate up to 2, 000, 000).
• Channel B: Used for logging output (fixed baud rate: 460, 800 for T5AI).

When connecting your board, make sure to select Channel A for flashing and Channel B for monitoring serial logs to ensure proper communication with your device.

Quick Tip: The T5E1 board features two serial COM ports:Channel A: Used for flashing firmware (baud rate dynamic, any baud rate up to 2, 000, 000).Channel B: Used for logging output (fixed baud rate: 460, 800 for T5AI).When connecting your board, make sure to select Channel A for flashing and Channel B for monitoring serial logs to ensure proper communication with your device.

Method 2: GUI Tool Flashing

If you prefer a graphical interface, you can use Tuya's flashing tools:

• Download the Tuya Flash Tool from the Tyutool GUI Tool
• Connect your board via USB
• Open the flash tool
• Select your firmware file (.build/bin/your_chat_bot_**QIO**.bin)
• Select the correct serial port
• Click "Flash" and wait for completion

Logging and Debuging

After flashing, you can monitor the device output to verify it's working:

tos.py monitor # And select the Channel B Serial Port for debug logging port

Or use any serial monitor tool (like screen, minicom, or PuTTY) with these settings:

• Baud rate: 460800
• Data bits: 8
• Stop bits: 1
• Parity:None
• Flow control: None

You should see boot messages and initialization logs.

Serial Logging will look something like:

How to Get a License Code

There are two methods to obtain a TuyaOpen license code:

Method 1: Free License Code Program

1. Create a product on the Tuya Developer Platform. Log-in to the Tuya Developer Platform and select a category to create a AI product based on your product type.

2. After creating the product, select the T5 module, then click "Add Custom Firmware" to upload the compiled QIO (production firmware) and UG (upgrade firmware). Alternatively, you can obtain an authorization code before uploading the firmware.

3. Click to claim 2 free authorization codes.

4. Select the License list. (During the product development phase, you can claim 2 free device licenses (valued at USD $4.5) for product development and debugging. After clicking "Claim, " the 2 licenses will be directly issued to you.)

Method 2: Purchase a License Code

Tuya Developer offers 2 free keys, when creating a new product, if you need more key license, you can always purchase additional one. It is per device, lifetime access key. As of time of writting it's about USD $2.15 per device license. And comes with AI access with unlimited token.

• Visit the Tuya License Purchase Page
• Select "TuyaOpen License Code"
• Complete the purchase
• You'll receive a UUID and AuthKey

Configuring the License Code

Once you have your UUID and AuthKey, you need to configure them in your project:

• Navigate to the application directory:

cd apps/tuya.ai/your_chat_bot/src

Open tuya_config.h in a text editor

• Open tuya_config.h in a text editor

Find and replace these lines:

• Find and replace these lines:

#define TUYA_DEVICE_UUID "your-uuid-here"
#define TUYA_DEVICE_AUTHKEY "your-authkey-here"

• Replace your-uuid-here and your-authkey-here with your actual UUID and AuthKey

Example:

#define TUYA_DEVICE_UUID "uuidxxxxxxxxxxxxxxxx"
#define TUYA_DEVICE_AUTHKEY "keyxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

• Save the file and rebuild:

tos.py build

• Flash the updated firmware:

tos.py flash

Method 2: Upload the AuthKey Using the GUI Tool

You can use the official Tuya license writing tool (Tyutool) to upload your AuthKey and UUID to the device quickly and easily.

1. Download the tool from the Tyutool Releases page.

2. Launch the Tyutool GUI application.

3. In Tyutool, select the correct serial port, set the baud rate (typically 115200), and choose your device model if needed.

4. Enter your device's UUID and AuthKey into the designated input fields under "Device Authorization Information."

5. Click "Write" to burn the authorization information to your device. Wait for the tool to show a successful write status.

6. Click "Upload" to write the license information to your hardware.

Important Notes:

• The license code must be TuyaOpen-specific. Regular Tuya device license codes will not work.
• Keep your license code secure and don't share it publicly.
• Each device needs its own unique license code.

Pairing to Smartphone

After flashing the firmware and configuring the license code, you can pair your device with the Tuya Smart Life app.

Step 1: Download the Smart Life App

Download the Tuya Smart Life app (or Tuya Smart app) from:

• Apple App Store
• Google Play Store

Step 2: Create or Login to Your Account

Step 3: Add Your Device

• Make sure your device is powered on and in pairing mode
• In the Smart Life app, tap the "+" (plus) button in the top right corner
• Select "Add Device" or "Add Manually"
• The app will scan for nearby devices

Step 4: Device Added

• Wifi connected, and start chatting with the AI using the hardawre.

Customization and Next Steps

Default AI Agent

This demo example comes with a default, basic AI agent with no specific character or custom prompt. It uses general-purpose AI models for conversation.

Advanced AI Agent Development

In a later chapter, we'll guide you through the process of setting up your own custom AI agent for more advanced control and workflow customization. For now, here's an overview of what you can do:

Customize AI Prompts:

• Modify the AI agent's personality
• Set specific roles and behaviors
• Configure response styles
• Customize AI Prompts:Modify the AI agent's personalitySet specific roles and behaviorsConfigure response styles

Configure AI Models:

• Switch between different LLMs (ChatGPT, Gemini, etc.)
• Adjust model parameters
• Set up custom model endpoints
• Configure AI Models:Switch between different LLMs (ChatGPT, Gemini, etc.)Adjust model parametersSet up custom model endpoints

Create Custom Workflows:

• Design multi-step agent workflows
• Add custom functions and tools
• Integrate with external services
• Create Custom Workflows:Design multi-step agent workflowsAdd custom functions and toolsIntegrate with external services

Develop Custom UI:

• Customize the LVGL interface
• Add new screens and interactions
• Implement custom animations
• Develop Custom UI:Customize the LVGL interfaceAdd new screens and interactionsImplement custom animations

Customizing Sensor and AI Hardware I/O Interactions

Since your device is multimodal, you can integrate a variety of sensor data—including text, numbers, audio, and images—directly into your AI workflows. By feeding this rich and diverse data into the LLM, you unlock the full power of advanced AI, enabling more intelligent, context-aware, and responsive agent hardware behavior. This flexibility allows you to build truly interactive and adaptive portable AI solutions tailored to your needs.

Resources and Documentation

TuyaOpen Open Source Project Links

• TuyaOpen Official Website- Main documentation and resources
• TuyaOpen GitHub Repository- Source code and issues
• TuyaOpen Quick Start Guide- Getting started tutorial

Frequently Asked Questions (FAQ)

• TuyaOpen FAQs and Answers – Common questions and solutions for development, licensing, cloud, and hardware issues

T5 Chip/Module Documentation

• Tuya T5 Chip Datasheet - Complete chip specifications
• Tuya T5-E1 (Onboard Antenna) Module Datasheet - Module with integrated antenna
• Tuya T5E1-IPEX (External Antenna) Module Datasheet - Module with external antenna connector

Cloud Development Documentation

• Cloud AI-Agent Development - Build custom AI agents
• Custom App Panel (Mini Program) Development - Create custom app interfaces
• Custom MCP Development SDK - Model Context Protocol SDK
• Custom MCP Cloud Configuration Documentation - Configure MCP in cloud

Resources for Advanced Development:

• Cloud AI-Agent Development Guide
• Custom MCP Development SDK
• Custom MCP Cloud Configuration
• Custom App Panel (Mini Program) Development

Hardware Resources

• Waveshare T5-E1-Touch-AMOLED-1.75 Wiki - Complete hardware documentation
• T5AI-Board Development Kit - Official Tuya development board
• T5AI-Core Minimal Core Development Kit - Minimal core board

Community and Support

GitHub Issues

• TuyaOpen GitHub Issues - Report bugs, suggest features, or seek help
• TuyaOpen Discord Community - Join the community chat
• Tuya YouTube Channel - Video tutorials and demos

Conclusion

Congratulations! You've successfully built your own portable AI agent using TuyaOpen and the Waveshare T5AI AMOLED 1.75" development board. You now have:

✅ A working AI agent device

✅ Voice interaction capabilities

✅ Beautiful AMOLED display

✅ Cloud connectivity

✅ Remote control via smartphone app

More Project Ideas

Now that you have a working AI agent, here are some project ideas:

Smart Home Assistant

• Control other Tuya devices
• Set reminders and alarms
• Answer questions about your home

Educational Device

• Interactive learning tool
• Language practice partner
• Homework helper

Custom Business Applications

• Customer service assistant
• Product information kiosk
• Interactive display

Entertainment Device

• Storytelling companion
• Game partner
• Music and media controlle

This is just the beginning! You can now customize your AI agent, add new features, integrate with other devices, and build amazing IoT projects.

Happy building! 🚀