MIT’s New Chip Gives Tiny Robots 3D Vision

Before a robot can navigate through the world around it, it needs some awareness of its surroundings. The best, most detailed information comes from 3D maps. These maps assist robots in planning collision-free paths to their destinations and in manipulating nearby objects. However, generating 3D maps requires a lot of equipment. Machines have to be loaded down with bulky, power-hungry, and expensive sensors and processing units to produce these maps.

Where does that leave the tiniest of robots? With subpar navigation systems, to put it lightly. But a new chip developed by MIT researchers could change that in the future. Their creation can generate 3D maps without the bulk of traditional systems, and it only uses about as much energy as a single LED.

The chip, called Gleanmer, was designed specifically for small autonomous systems where every milliwatt matters. Tiny drones inspecting industrial equipment, robots searching confined spaces, and lightweight augmented reality headsets all face the same challenge: they need detailed spatial awareness, but they cannot afford the power budget required by conventional mapping hardware. According to the researchers, Gleanmer consumes less than 6 milliwatts while constructing detailed 3D occupancy maps in real time.

Traditional mapping systems typically represent the world using voxels, which are essentially three-dimensional pixels. While effective, voxel maps require large amounts of memory and computational power. The MIT team took a different approach by representing obstacles and free space using Gaussian ellipsoids. These flexible shapes can stretch and adapt to match the geometry of objects much more efficiently than rigid cubes. A single Gaussian can often describe an area that would otherwise require many voxels, dramatically reducing storage requirements.

The hardware is paired with an algorithm known as GMMap, which was also developed by the researchers. One of its key advantages is that it can generate these Gaussian representations from depth images in a single pass. Conventional techniques often need to revisit image data multiple times while comparing large numbers of pixels. GMMap instead assumes neighboring pixels belong together, allowing it to process data as it arrives and discard it immediately afterward. This means the chip never has to store an entire image in memory at once.

Another challenge in 3D mapping occurs when a robot views the same object from multiple angles. This often creates overlapping representations that waste memory. Gleanmer addresses this by merging overlapping Gaussians directly, without revisiting the original image data. This makes for a smaller map that requires less processing power to maintain.

The researchers co-designed both the software and hardware so that frequently accessed map data remains in small, fast memory located directly beside the processing units. This reduces the need to constantly retrieve information from larger, more power-hungry memory systems. The approach allows Gleanmer to construct maps from 640×480 depth images at more than 88 frames per second while maintaining its remarkably low power consumption.

Beyond robotics, the researchers believe the technology could find applications in wearable augmented reality devices and potentially even future AI systems that need efficient ways to represent complex spatial information.