Pis with Eyes for Affordable Optical Tracking

Multi-camera systems are now an essential tool for tracking markers in 3D space, playing a critical role in various applications such as virtual reality (VR) and motion capture. These systems use a network of synchronized cameras strategically placed around a designated area to capture the movement of markers, allowing for the precise reconstruction of an objects' position and orientation in three-dimensional space. By using triangulation and sophisticated algorithms, these systems can accurately track the movements of objects or individuals in real-time, providing invaluable data for applications ranging from entertainment to scientific research.

In VR, multi-camera systems are necessary for creating immersive and interactive experiences. These systems track the user's movements and interactions with precision, allowing them to seamlessly integrate their movements into the virtual environment, enhancing the sense of presence and immersion. Multi-camera systems have also proven to be instrumental in motion capture, capturing the intricate movements of actors and performers with exceptional accuracy. These systems are widely used in the production of movies, animation, and video games, allowing for the creation of lifelike digital characters that mimic the motions and expressions of the actors.

However, the implementation of optical tracking systems comes with significant challenges, including high costs and complexities that limit their practicality in many use cases. These systems often require a substantial investment in specialized hardware, including high-resolution cameras, high-powered computational devices, and intricate calibration equipment. Moreover, the installation and maintenance of the camera network demands skilled technicians and meticulous calibration procedures, adding to the overall complexity and cost of the system.

Over at Seneral, a team is working to change this present situation with an optical tracking system called AsterTrack. AsterTrack is built around low-cost hardware with the goal of providing the same type of capabilities as professional systems in a more user-friendly and accessible way. Towards this goal, Raspberry Pi Zero computers are being leveraged in conjunction with some highly-optimized algorithms written in low-level languages.

The AsterTrack tracking system consists of three primary components — cameras, a controller, and a configurator. A set of three cameras can track either active or passive markers using an OV9281 image sensor and a Raspberry Pi Zero. The controller synchronizes the cameras, such that the two-dimensional images of markers captured by each camera can be translated into a location in three-dimensional space by the configurator, which runs on an external computer. This computer runs the algorithms that determine the final tracking result and also interfaces with other devices.

It was through the development of low-level assembly language functions that fully utilize the Raspberry Pi’s quad-core CPU that AsterTrack was made performant. In this way, the system can track objects in 0.9 megapixel images at a rate of 120 frames per second. And that speed does not come at the expense of accuracy — global shutter cameras with external synchronization ensure that the configurator has accurate information from which to infer the position of a tracked object.

Nearly any everyday object can be tracked by affixing either retroreflective tape or infrared LEDs to it. After a short calibration process, which is promised to be user-friendly, AsterTrack is ready to go.

If it works as well as promised, AsterTrack could open up lots of new opportunities for optical tracking and motion capture where the cost had previously been prohibitive. However, the system is presently under development, so you cannot order one for yourself just yet. The developers do expect to start a crowdfunding campaign before the end of 2023, so be sure to sign up for updates if you want to be among the first to score an AsterTrack system.