Gesture Spotify Controller: Control Music with Your Hand

What is this project?

Gesture Spotify Controller is a hand-gesture music remote that runs entirely on an Arduino UNO Q — no cloud, no video streaming, no companion app watching you. Hold up your hand, make one of four gestures, and your Spotify playback responds: Play, Pause, Next Track, Previous Track. The camera feed never leaves the board. All the inference happens on-device.

Why I built it

I wanted a project that actually earned its "edge AI" label instead of just being a webcam hooked up to a cloud API. I went through a few ideas first — a recycling sorter, a posture monitor, a room air quality sensor, a cable identifier, a desk sentinel — but each one either needed data collection that wasn't realistic in my timeframe, or felt like a gimmick that wouldn't hold anyone's attention past the first demo. Gesture control solved both problems: it's genuinely useful (who hasn't wanted to skip a track without touching their phone mid-workout or mid-cooking), and it's a clean showcase of real-time, fully local computer vision on a tiny board.

How it works

• Seeing the gesture — A USB webcam feeds frames to the UNO Q. Edge Impulse's FOMO (Faster Objects, More Objects) model — trained on four gesture classes (Play, Pause, Next_Track, Previous_Track) plus background negative frames — detects and classifies the gesture directly on-device, frame by frame.

• From detection to action — App Lab's VideoObjectDetection module runs the model and fires an on_detect_all callback whenever a gesture is confidently recognized. That callback is debounced so a held gesture doesn't spam repeat commands.
• Crossing the board — The Linux side (running the Python inference) talks to the microcontroller side over Bridge RPC, so the MCU can drive the Modulino Pixel and Buzzer for instant physical feedback — a flash of color and a beep confirm the gesture was caught, before the Spotify command even completes.

• Reaching Spotify — The board calls the official Spotify Web API (via spotipy) to control whatever device is actively playing — a phone, a laptop, a speaker — anywhere on your Spotify Connect account. Authentication happens once, on a laptop, to generate a token; that token is copied to the UNO Q, which then runs headless with no browser and no dependency on any other machine being powered on.

The result: you can wave your hand at a little board sitting on your desk, and music halfway across the room changes, with the recognition, feedback, and decision-making happening entirely inside the UNO Q.