MIT’s New Tech Gives Underwater Robots Perfect 3D Vision in Murky Waters

Just a tiny fraction of one percent of the ocean floor has been visually surveyed by cameras due to the challenging conditions encountered there. Many factors make ocean mapping difficult, including the pressures exerted by deep water, wireless communication issues, and turbidity that obscures vision. These problems also plague commercial operations involving underwater infrastructure construction and maintenance.

Researchers at MIT have just developed a new imaging system that addresses one of these problems head-on. Thanks to their efforts, future underwater vehicles will be able to get a clear view of their surroundings, no matter how murky the water may be.

The new system, called Sonar-MASt3R, combines data from optical cameras and sonar sensors to generate detailed three-dimensional maps of underwater environments in real time. This technology is designed specifically for situations where visibility is severely degraded by suspended sediment and other particles.

Underwater vehicles have traditionally relied on either cameras or sonar. Cameras provide detailed color and texture information, but their usefulness quickly diminishes in dark or cloudy water. Sonar, on the other hand, performs well regardless of visibility conditions by measuring reflected acoustic waves to determine the shape and location of nearby objects. However, sonar images generally lack the fine visual details that cameras can provide.

Sonar-MASt3R combines the strengths of both approaches. The system builds upon an existing image-processing framework called MASt3R, which can rapidly estimate the relative depth of objects from ordinary camera images. One limitation of MASt3R is that it cannot determine absolute scale. A reconstructed object may appear correctly shaped, but the system does not inherently know whether it is a few centimeters away or several meters away. Sonar measurements solve that problem by providing precise distance information that can be used to scale the camera-derived model accurately.

The researchers describe the technique as similar to combining a dolphin’s echolocation with a sea turtle’s close-range vision. Sonar first generates a coarse map of the surrounding environment, allowing an underwater robot to identify obstacles and points of interest even when visibility is poor. The vehicle can then safely approach those locations and use its cameras to capture higher-resolution imagery that refines the map with visual detail.

To test the technology, the team built a controlled underwater environment containing objects such as a coffee mug, a packing crate, and a small boulder. A robotic arm equipped with both an underwater camera and an imaging sonar sensor swept across the tank while the researchers varied water turbidity by stirring up sediment. The system was evaluated across visibility levels ranging from clear water to conditions so cloudy that conventional cameras were essentially blind.

The results showed that Sonar-MASt3R could produce more accurate three-dimensional reconstructions than previous opto-acoustic fusion techniques while also resolving smaller, centimeter-scale features. Even when the cameras could not directly see objects through the murk, the sonar-generated map enabled the robotic arm to navigate safely toward them and gather the visual information needed for a detailed reconstruction.

The researchers envision applications ranging from scientific exploration and deep-sea archaeology to underwater construction, infrastructure maintenance, and the recovery of hazardous objects such as unexploded underwater mines. Future testing will move beyond laboratory tanks and into natural underwater environments, where the team believes the system may perform even better due to reduced acoustic reflections and interference.