Sim-to-Real in Unmanned Surface Vehicle Control: A System Identification-Based Approach for Enhanced Training Environments

Addressing the issues of safety, efficiency, and effectiveness in the training of reinforcement learning strategies, this paper proposes a simulation training environment construction method based on system identification, facilitating the sim-to-real transfer from the simulation training environment to the application environment. A relatively accurate unmanned surface vehicle (USV) dynamics model is obtained using system identification methods, with data collected from the real operational environment. Based on this dynamics model, an efficient and realistic simulation training environment is constructed. The interaction samples obtained from this simulation environment are used for the training of strategies, and the resulting strategy model can be directly deployed to the application environment. Simulation in the Unity3D platform demonstrate that the constructed simulation training environment has high interaction efficiency, and the strategy model obtained maintains high performance in the application environment.