Pole Position

Mixing the physical world with virtual elements is an idea that has been around for quite some time, but the potential of this technology has still not yet fully arrived. Once it does, it could help to unleash creativity and collaboration by creating workspaces that allow for rapid prototyping and ideation that could never exist in the real world. These systems are also expected to transform entertainment, where they might, for example, add interactive elements to board games, or superimpose a video game on an ordinary, uninstrumented surface.

Virtual and augmented reality headsets have advanced tremendously in recent years, which makes the visual and auditory experience very convincing for applications such as these. But when it comes to the interactive components, much work still needs to be done. In particular, the human body — and other physical objects — must be accurately tracked so that they can be integrated into the mixed reality experience.

While a number of very accurate tracking devices do exist today, they all suffer from one problem or another. Camera-based systems tend to be highly accurate, but they can easily get confused when objects get between them and what they are supposed to track. Moreover, these systems are expensive and can be a challenge to set up initially, which makes them impractical for many use cases. Other sensing options make use of inertial measurement units to track objects, but these are less accurate and suffer from drifting errors over time.

University of Michigan and Shanghai Jiao Tong University researchers have just introduced a new option called MagDesk that appears to solve these problems — so long as the application is confined to a tabletop area, anyway. MagDesk uses magnetometers to sense the three-dimensional position of magnets that are embedded in objects to be tracked. This low-cost system does not require line-of-sight to the sensors, and it was demonstrated to have millimeter-level accuracy.

The novel tracking system makes use of an array of 112 low-cost magnetometers that are installed underneath a normal tabletop. These sensors, arranged in a 7x16 array with 10 cm spacing, detect magnetic fields from embedded magnets and track them in five degrees of freedom (3D position and 2D orientation) with millimeter-level precision. The system has a working range of up to 600 mm above the table surface and spans a 1,750 mm x 950 mm area.

A set of seven Arduino-compatible development boards collect data from the magnetometers and stream it to a computer for processing. The software pipeline begins with an initialization process, where the system calibrates to account for background magnetic fields and static objects. During operation, an analytical solver, leveraging a dipole model and gradient descent, calculates the magnet positions. This is enhanced by an adaptive filtering pipeline combining Kalman filtering and moving averages, which dynamically adjusts parameters based on magnet size, speed, and height for optimal tracking accuracy.

MagDesk was evaluated in a series of experiments that measured its tracking accuracy and performance in single- and multi-magnet scenarios. A benchmarking setup utilized HTC Vive Trackers as ground truth references, with magnets rigidly attached to the trackers and infrared-based SteamVR Base Stations providing spatial alignment. For single-magnet tracking, performance was tested at various fixed locations and heights above the table, capturing both position and orientation errors under controlled conditions. The system achieved high precision, with errors as low as 2.49 mm and 0.72 degrees near the table surface, and robust tracking up to 600 mm above the table. Multi-magnet tracking was tested with magnets in close proximity, addressing challenges such as overlapping magnetic fields. Using a specialized two-magnet solver, the system improved accuracy in these scenarios, maintaining a positional error of 11.18 mm at a 30 mm separation between magnets.

The team showed that their system supports applications such as 3D drawing and augmented reality tabletop gaming, demonstrating its ability to enable tangible interactions in home and office environments.