Pixel-Pets: An AI-Assisted Virtual Pet Family for M5Stack

The Spark: A Father-Son Collaboration

Pixel Pets started as a way to bridge retro nostalgia with modern maker tech through a shared family activity. Justus (10) acted as the chief designer and product owner, defining how the pets should feel, react, and play. I served as the "technical wingman, " translating his ideas into constraints for our AI development process.

What is Pixel Pets?

Pixel Pets is an open-source virtual pet ecosystem designed for the M5Stack hardware family. It consists of three main pet variants—Muffin (CoreS3 + LLM), Visu (CoreS3), and Goo-Goo (Core2)—alongside a pocket-sized accessory called Pip. The project brings digital companions to life with 11 different moods, mini-games, and a world-aware system that reflects the user's real environment.

How it Works

The technical foundation of the project is built on several interconnected modules:

AI-Assisted Coding: Every line of firmware was written by AI assistants (primarily Claude) under our direction, making it a unique showcase of modern "co-authored" development.

• AI-Assisted Coding: Every line of firmware was written by AI assistants (primarily Claude) under our direction, making it a unique showcase of modern "co-authored" development.

World Awareness: Upon booting, the devices use IP geolocation to pull real-time weather, sunrise/sunset times, and moon phases from Open-Meteo to adapt the pet's background.

• World Awareness: Upon booting, the devices use IP geolocation to pull real-time weather, sunrise/sunset times, and moon phases from Open-Meteo to adapt the pet's background.

Inter-Device Communication: We used ESP-NOW to allow the pets to interact with each other and to enable the Pip accessory to "throw" treats to a home pet with sub-200ms latency.

• Inter-Device Communication: We used ESP-NOW to allow the pets to interact with each other and to enable the Pip accessory to "throw" treats to a home pet with sub-200ms latency.

Interactive Input: Pets respond to touch zones on the screen, voice commands via an offline LLM (on Muffin), and physical movement via the IMU for petting or shaking.

• Interactive Input: Pets respond to touch zones on the screen, voice commands via an offline LLM (on Muffin), and physical movement via the IMU for petting or shaking.

Parental Controls: A built-in web server allows parents to set daily play-session limits to ensure a healthy balance.

• Parental Controls: A built-in web server allows parents to set daily play-session limits to ensure a healthy balance.

Visuals & Interaction

The project is highly visual, featuring custom pixel art and fluid animations.

The whole hardware family: Muffin, two Goo-Goos, and the Pip accessory.

A close-up of the Bear pet on a Goo-Goo (Core2) device.

You can even see the pets in action in our 10-second demo video.