URoBench: Comparative Analyses of Underwater Robotics Simulators from Reinforcement Learning Perspective

In an effort to standardise the evaluation of Reinforcement Learning (RL) algorithms across different simulators of underwater robots, this paper introduces a benchmark framework, URoBench, to test the capabilities of simulators in RL- related tasks of underwater robotics. This framework is characterised by its modular architecture. By abstracting these components, the framework allows for integration with various simulators through different interfaces. To verify the framework, we tested the performance of three typical underwater robotics simulators (HoloOcean, Dave, and Stonefish) in RL conditions. Also, the usage effectiveness of the computation resources of these three simulators was compared. The verification result demonstrates the feasibility of incorporating this benchmarking framework into different simulators, facilitating consistent and comparable assessments. This benchmarking framework stands to provide a common assessment method for underwater robot simulators, standardising the development and simulation of RL for marine robotics.

I. Introduction

Within the robotics field, Reinforcement Learning (RL) has demon-strated significant capabilities in various applications, enabling drones for autonomous racing, allowing quadrupedal robots with robust locomotion in diverse terrains, and equipping robots with the skills to manipulate objects [1]–[3]. RL has also shown capable of addressing the challenges encountered by underwater robots due to its adaptability and learning capabilities. Its applications in this domain encompass autonomous navigation and exploration [4]–[6], position tracking [7], manipulation of underwater objects [8], [9], and adaptive control in dynamic underwater environments [10].