Why Is That Robot Wearing a VR Headset?

It may seem like a strange idea at first, but when you think about it some more, Ludic Worlds’ robot design actually makes a lot of sense. Ludic Worlds created a robot that uses a Meta Quest 3 headset as the brains of the operation. And why not? These headsets are equipped with powerful processing units and loads of sensors that can handle spatial mapping and positional tracking tasks with ease.

Since the headset handles so much, the body of the robot is pretty basic. It was made with LEGO bricks, an L298N motor driver, DC motors, six AA batteries for power, and an ESP32 development board to control the motors. The headset simply rests on top of the robot body.

The Quest 3 is by no means a hobbyist’s development board, so it was not designed for interfacing with external devices. You won’t find GPIO pins or other such interfaces. Ludic Worlds got around this limitation in the easiest way possible by using Wi-Fi to communicate with the ESP32. This is not as fast as a hardwired connection, but for a relatively slow-moving robot, there isn’t enough delay to matter — especially considering the decision to use UDP packets for communication.

Ludic Worlds’ plan was to give simple spoken commands to the robot, then have it autonomously drive to the requested location. This was accomplished by using Whisper to transcribe the user’s voice. An application developed with Unity then maps this location to a predefined spatial anchor in the room. The headset’s cameras and other sensors are then used to create a plan for navigating to the destination. The navigational plan is translated into motor movements, which are transmitted to the ESP32 for execution.

Using a Meta Quest 3 as a robot controller may seem like an odd thing to do, but it saves a lot of time assembling and debugging hardware. Check out the video below to see this little robot in action.