NASA’s Favorite Controller Is the Secret to Cheap Robot Training

SpaceMouse controllers are one of the most useful computer input devices for an engineer. Whether one is designing a custom PCB or inspecting a 3D model, the SpaceMouse makes it easy to navigate. And productivity gains aside, these devices are just plain cool. Did you know that the first SpaceMouse was used to manipulate a robotic arm on the Space Shuttle Columbia? That makes you want one on your desk even more, doesn’t it?

Hardware hacker Nikodem Bartnik has a SpaceMouse and generally uses it for navigating 3D models. However, knowing the history of this device, that seemed a little too ordinary. So, Bartnik decided to put this SpaceMouse to better use by hooking it up to a robot arm.

The arm in question was the Seeed Studio SO-ARM101, a 3D-printed, low-cost robotic arm that sells for roughly $260 to $300 once motors and electronics are included. Bartnik’s goal was to find out if a 3Dconnexion SpaceMouse could replace a second “leader” robot arm in a traditional leader-follower training setup. If so, hobbyists could save both money and bench space when collecting data to train machine learning models.

The SpaceMouse offers six degrees of freedom, allowing translation and rotation in 3D space. The SO-ARM101, however, only has five degrees of freedom. That mismatch immediately caused problems. Bartnik first attempted to use inverse kinematics (IK) to map the SpaceMouse’s movements into 3D positions for the robot’s gripper. But with one extra degree of freedom unaccounted for, the arm frequently behaved unpredictably.

Rather than abandoning the experiment, he switched strategies. Instead of using IK, he directly mapped specific SpaceMouse motions — tilting, twisting, and sliding — to individual joints on the arm. This simpler approach proved far more stable and gave him usable teleoperation control.

On the software side, Bartnik used the Lerobot library from Hugging Face but had to fork it and write custom code to integrate SpaceMouse support. He initially ran the setup on an NVIDIA Jetson Orin Nano, but later migrated to a desktop Ubuntu PC due to limited USB ports for dual camera feeds. Even then, he encountered lag during data recording. The teleoperation loop ran at 60 Hz, but the recording script choked on the camera processing load. Optimizing the SpaceMouse data queue ultimately resolved the responsiveness issue.

With the system working, Bartnik recorded 30 training episodes of a simple task: picking up a LEGO brick and placing it into a container. Unlike a leader-follower arm — which requires virtually no practice — the SpaceMouse required 15 to 30 minutes of training before smooth control was possible. He then trained an ACT (Action Chunking with Transformers) policy on an NVIDIA RTX 4090, a process that took about 45 minutes.

The trained model could autonomously identify, grasp, and deposit the LEGO brick as intended. While the motions were less fluid than those trained with a second robot arm, the experiment proved that “space-age” input hardware can indeed train desk-scale robotics.