BIER Lets Uncrewed Underwater Vehicles Make Better Decisions Faster, Researchers Find

Researchers at Flinders University, École Nationale Supérieure de Techniques Avancées Bretagne (ESTA Bretange), and the French Naval Group Research Center have come up with a new bio-inspired approach to creating autonomous uncrewed underwater vehicles (UUVs) — and say their system, the Biologically-Inspired Experience Replay (BIER) method, could be applied to improve other adaptive control systems.

“The outcomes of the study demonstrated that BIER surpassed standard Experience Replay methods, achieving optimal performance twice as fast as the latter in the assumed UUV domain," explains first author Thomas Chaffre, PhD, of the team's work. "The method showed exceptional adaptability and efficiency, exhibiting its capability to stabilize the UUV in varied and challenging conditions."

BIER differs from traditional alternatives by using two memory buffers, one of which is effectively a "short-term memory" focused on recent state-action pairs and the other of which exists to emphasize positive rewards. Compared to rival deep reinforcement learning (DRL) approaches, BIER delivered better results with weak data or degraded performance — using incomplete but nevertheless valuable recent experiences to make decisions about how to approach new scenarios.

"The proposed method was evaluated through simulated scenarios in a ROS [Robot Operating System]-based UUV Simulator, progressively increasing in complexity," the researchers explain. "These scenarios varied in terms of target velocity values and the intensity of current disturbances. The results showed that BIER outperformed standard Experience Replay (ER) methods, achieving optimal performance twice as fast as the latter in the assumed UUV domain."

"We have argued that the proposed method can be considered in processes where proportional feedback control can be derived, which represents the majority of UUV applications," the researchers conclude. "Future work shall explore the extent to which this work could generalize to other domains, for the control of distinct types of vehicles operating under various environmental conditions."

The team's work has been published under open-access terms in the journal IEEE Access.