A DIY Raspberry Pi-Based Mocap System to Rival Hollywood

Mocap (motion capture) has revolutionized the film and video game industries. But despite being in widespread use for a few decades now, it is still wildly expensive and only accessible to studios with serious budgets. That may change soon though, because YouTuber Made By Dennis has built a Raspberry Pi-based DIY mocap system to rival what’s in Hollywood.

Mocap basics are easy enough to understand: multiple cameras around the room track markers placed on actors or objects. When at least three cameras can see a marker, a computer can calculate that marker’s position in 3D space as it moves. Those then become reference points for rigging digital “puppets” in virtual space.

But on a technical level, it is much, much more complex than that. The cameras need to record at a high framerate with minimal motion blur and they need to stay in sync for the position calculations to be accurate. And in this case, the complete system also needs to be affordable.

To meet those requirements, Dennis designed his own cameras. Each does its own image processing, thanks to a built-in Raspberry Pi Compute Module 5. They record video through camera modules at 120fps and do so without an IR filter. That meant that Dennis could integrate 2.5kW infrared LED strobe lights to brightly illuminate the retroreflective markers, “freezing” motion to prevent blur. Each camera outputs the in-frame coordinates of detected markers to a central computer, which does the calculations to yield the 3D coordinates of all of the markers.

And it does that in real-time, which means this system works for applications like closed-loop control of drones and robots.

Each camera needs to figure out where the markers are in the frame and needs to do so 120 times every second — all while staying in sync with the other cameras. That’s far faster than could be achieved with normal software approaches, so Dennis made kernel-level tweaks to optimize throughput. He used many tricks, including starting the processing of each frame before the camera even finishes transferring the data to the Raspberry Pi, to achieve the desired performance.

How well did that all work? We don’t actually know yet, because Dennis is saving the big reveal for a future video. But the system is looking incredibly promising so far and I’m confident that it will be worth the wait.