Reinforcement Learning Helps Autonomous Vehicles Safely Navigate the Briny Depths
Autonomous exploration of "Earth's final frontier" becomes easier by borrowing a similar approach to that of large language models (LLMs).
A team of researchers from the Institut de Ciènciesdel Mar (ICM), Monterey Bay Aquarium Research Institute (MBARI), Barcelona Tech, the Universitat de Girona, and the Barcelona Supercomputer Center (BSC) has come up with a reinforcement learning approach to locating and tracking objects underwater — and aims to deploy the technology in future autonomous vehicles.
"This [reinforcement] learning allows us to train a neural network to optimize a specific task, which would be very difficult to achieve otherwise," says lead author Ivan Masmitjà of the team's focus on the approach, which is similar to that underpinning natural language processing systems like large language models (LLMs). "For example, we have been able to demonstrate that it is possible to optimize the trajectory of a vehicle to locate and track objects moving underwater."
For all that creating reliable autonomous vehicles which can traverse the skies and roads is challenging, doing the same underwater brings all-new difficulties. Crewed missions to extreme depths are not without danger, as recent events have shown, and autonomous vehicles have the potential to fill in valuable gaps in data — but only if they can navigate beneath the waves and correctly spot objects for navigation, observation, or avoidance.
The team's work used range acoustic techniques to estimate the position of an object using measurements taken at different points, an approach which is normally accurate only in the region where the measurements were originally taken — then applied reinforcement learning to create a generalized trajectory for the autonomous vehicle to follow.
"[This] will allow us to deepen the study of ecological phenomena such as migration or movement at small and large scales of a multitude of marine species using autonomous robots," says co-author Joan Navarro of the research's likely impact. "In addition, these advances will make it possible to monitor other oceanographic instruments in real time through a network of robots, where some can be on the surface monitoring and transmitting by satellite the actions performed by other robotic platforms on the seabed."
The approach has proven its worth in both simulation and at sea with vehicles including the Sparus II Autonomous Underwater Vehicle (AUV) — and the team is now working on studying the applicability of the same algorithms to more complex missions, including those involving multiple autonomous vehicles working collaboratively.
The team's work has been published under open-access terms in the journal Science Robotics.
Main article image courtesy of Iqua Robotics.