Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Developing an autonomous collision avoidance system for an Unmanned Surface Vehicle (USV) operating in unpredictable environments is a challenge. This is especially true in crowded inland waterways where vessel density is high, and the navigable area is limited. Ships should be able to make autonomous decisions to avoid collisions with other ships. Additionally, reports indicate that the majority of ship collision accidents are caused by various human decision-making errors. In this study, we propose an unmanned surface vehicle collision avoidance method based on a deep reinforcement learning algorithm. This method utilizes a Deep Neural Network (DNN) to directly map the state of the ship to the steering angle. This DNN is trained over USV in rich encountering situations using the Soft Actor-Critic (SAC) algorithm, which can automatically adjust temperature coefficients. To better assess the collision risk of ships, this paper constructs two types of ship domains: the avoidance domain and the collision domain. We validate the proposed collision avoidance method in three typical encounter scenarios: head-on encounters, crossing encounters, and overtaking encounters. Simulation results demonstrate that the USV can safely avoid target ships and reach its destination in various encounter situations.