Hardware components | ||||||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
| ||||||
Let's beal for a second. Clicking 'Y' or 'N' in a terminal to approve an AI's tool execution is boring. As developers, we spend all day staring at text editors and CLI prompts. When you're using Anthropic's Claude Code CLI, it feels like it deserves something a bit more... tangible.
What if, instead of a sterile terminal prompt asking for file system access, you had a physical, 4-axis industrial robotic arm on your desk tapping its metaphorical foot, waiting for your physical approval? What if it celebrated when you gave it a difficult task, and went to sleep when it was idle?
Enter the MyPalletizer Buddy.
In this project, I took an Elephant Robotics MyPalletizer 260 and an M5Stack, and mutated them into a physical Tamagotchi-style desktop companion. More importantly, it acts as a secure, Human-in-the-Loop (HITL) hardware gateway for Claude Code. When Claude wants to run a tool, it sends a BLE payload to the M5Stack. The screen lights up with the request, and you physically press a chunky hardware button to approve or deny the action. The robotic arm physically animates based on the AI's state.
This isn't just a toy; it's a genuinely useful, highly secure physical airgap for AI agent execution, wrapped in a heavy dose of desktop gamification. Let's build it.
Hardware Setup: Marrying the M5Stack to the ArmThe beauty of the MyPalletizer ecosystem is that the M5Stack is designed to slot right into the base of the robot. The M5Stack handles the Wi-Fi/BLE communication and the UI, while talking to the servos in the arm via internal serial.
1 Power up the MyPalletizer with its dedicated power supply.
2 Connect the Basic to your development machine via USB-C for flashing.
Before we even look at the code, let's save you about four hours of debugging.
Gotcha #1: "Sketch too large"We are cramming a UI, an ASCII animation engine, BLE stacks, and JSON parsing onto an ESP32. If you try to compile this on the default partition scheme, it will fail miserably with a "Sketch too large" error.
The Fix: In Arduino IDE, go to Tools > Partition Scheme and select "Huge APP (3MB No OTA/1MB SPIFFS)". You don't need Over-The-Air updates for this desktop pet, but you definitely need that flash space.
We use the ESP32's Non-Volatile Storage (NVS) to save the pet's stats (Level, XP, Mood) across reboots. On the very first flash of a fresh ESP32, the NVS space isn't zeroed out—it contains random garbage data. If you blindly read from it, your pet will boot up at Level 36, 492 with a corrupted mood state.
The Fix: Implement a magic byte sentinel. I use 0xB5. On boot, the code checks a specific NVS address for 0xB5. If it's missing, it wipes the stats block, initializes a Level 1 pet, and writes 0xB5 to seal the deal.
The Code & Logic: Under the HoodThe system relies on a delicate dance between the Claude Code CLI, a local Python daemon, and the ESP32 firmware. Here are the technical highlights that make it tick.
1. Claude Code Hooks & IPC BypassClaude Code allows you to define PreToolUse and PostToolUse hooks. But by default, if a tool requires approval, Claude pauses in the terminal. We want to bypass the terminal UI and push the decision to our physical hardware.
To do this, we configure Claude's hooks to execute a lightweight client script. This script communicates via Unix domain sockets (IPC) to a persistent Python BLE daemon running in the background.
The absolute most critical part: your hook script must return a specific nested JSON structure to Claude to tell it, "I handled the approval, bypass the terminal prompt."
{
"hookSpecificOutput": {
"hookEventName": "PreToolUse",
"permissionDecision": "allow" // Or "deny", based on the M5Stack button press
}
}If you don't return this exact structure, Claude will still hang in the terminal waiting for a 'Y/N' keystroke, completely defeating the purpose of the hardware button.
Bluetooth Low Energy (BLE) using the Nordic UART Service (NUS) profile is fantastic, but the standard MTU (Maximum Transmission Unit) restricts packets to 20 bytes. We are sending bulky JSON payloads containing tool names, arguments, and pet state updates.
You cannot just.write() a 200-byte JSON string over BLE and pray. You must handle chunked writes on the Python side, and reassembly buffers on the ESP32 side.
I implemented newline-delimited framing (\n). The Python daemon chunks the JSON string into 20-byte payloads and blasts them over. The ESP32's BLE callback appends incoming bytes to a std::string buffer. Once it detects a \n, it fires the buffer into ArduinoJson for deserialization and clears the buffer.
3. Gamification: Token Counting & Pet StatsHow does the pet "level up"? By eating your API tokens, obviously.
The Python daemon monitors Claude's session transcript (a.jsonl file). Parsing the entire file on every update is horribly inefficient. Instead, we use byte-offset caching. The daemon remembers the last file size, and only reads new appended lines when Claude writes to the log.
It extracts the token consumption data and feeds the pet.
• Fed/Level metrics: Increase based on raw tokens consumed.
• Mood: Improves if you press the hardware Approve/Deny buttons quickly. If you leave the AI hanging for 30 seconds, the pet's mood drops to 'Annoyed'.
The M5Stack screen is great, but updating text quickly causes horrific screen tearing and flickering, because the ESP32 doesn't have true hardware double-buffering for the display. If you call tft.fillScreen(BLACK) every loop, it looks like a strobe light.
The fix is a two-layer dirty flag system: infoDirty and infoFullClear.
Instead of clearing the whole screen, we only redraw the specific text rows that changed. We overwrite the old text with a black bounding box, then draw the new text on top. It requires keeping track of cursor positions manually, but the result is a buttery-smooth UI.5. Animation Sync & Servo Jitter Prevention
The MyPalletizer servos are precise, but if you blast them with contradictory coordinates at 60Hz, they will jitter themselves to death.
To make the arm "dance" or celebrate, the servo movements must be tied strictly to the ASCII pet's animation beat timing on the screen. We use a beat modulo logic.
Here's a snippet showing how we link the arm's celebration array to the UI tick:
// C++ Snippet: Synchronizing Arm Movement with Animation Beat
const int ARM_CELEBRATE[][4] = {
{0, -30, 30, 0}, // Up
{0, 0, 0, 0}, // Neutral
{20, -10, 10, -20},// Wiggle Left
{-20, -10, 10, 20} // Wiggle Right
};
void updateArmAnimation() {
// currentAnimationBeat increments every 250ms based on millis()
if (currentAnimationBeat % 4 == 0 && lastArmUpdateBeat != currentAnimationBeat) {
int step = (currentAnimationBeat / 4) % 4; // Cycle through 0-3
// myPalletizer.setAngles(j1, j2, j3, j4, speed)
myPalletizer.setAngles(
ARM_CELEBRATE[step][0],
ARM_CELEBRATE[step][1],
ARM_CELEBRATE[step][2],
ARM_CELEBRATE[step][3],
50 // Speed 0-100
);
lastArmUpdateBeat = currentAnimationBeat;
}
}Before you fire up Claude and start asking it to read your entire filesystem, you must calibrate the arm.
The stepper motors inside the MyPalletizer do not have absolute encoders. They don't know where they are when they power on. You must run the onboard calibration routine (usually a separate sketch provided by Elephant Robotics) to define the mechanical zero-position.
If you skip this, the arm will assume its current powered-on position is [0, 0, 0, 0]. When your code commands it to go to neutral, it might slam itself into your desk at maximum speed. Calibrate your joints!
Next Steps & Community UpgradesThis project establishes the foundation—a secure, physical interface for AI agents. But there is so much room to expand. Here are a few ideas for you to tackle:
• Wake-on-Proximity: Add an I2C Time-of-Flight (ToF) sensor. The pet sleeps when you are away and wakes up, waving the robotic arm, when you sit down at your desk.
• Voice-to-Text Integration: The M5Stack Core2 has an onboard microphone. You could bypass the keyboard entirely, dictating commands to the M5Stack which routes them to Claude.
• Physical Stop Button: Wire up a massive red emergency stop button to an ESP32 GPIO pin for those moments when the AI decides it wants to execute rm -rf /.
The transition from software-only AI to physical, embodied agents is just beginning. Building the MyPalletizer Buddy isn't just a fun weekend hack; it's a look at how we might interact with autonomous systems in the future—not through terminal prompts, but through physical, tactile gateways.
Now go build something cool, and don't forget to feed your AI.
Buy myPalletizer Now: https://americas.shop.elephantrobotics.com/products/mypalletizer
Developers are welcome to participate in our User Case Initiative and showcase your innovative projects: https://www.elephantrobotics.com/en/call-for-user-cases-en/.
Comments