The USV Path Planning Based on an Improved DQN Algorithm

Research on the path planning algorithm of USV (unmanned surface vessel) has received widespread attention. Among them, the algorithm based on the combination of deep learning and reinforcement learning achieves better performance. However, in the complex water environment, this algorithm is prone to overestimate the action value, which will lead to path planning deviation. In this paper, we propose a Double DQN algorithm to optimize obstacle avoidance and path planning, which based on deep Q network (DQN) algorithm. The Double DQN algorithm will decouple the target Q value’s action selection and action evaluation, and index the action value corresponding to the maximum Q value through the current network, then input the selected action value into the target network to calculate the target Q value. As a result, the overestimation is reduced, and path deviations are corrected. The simulation results show that the Double DQN algorithm has a good performance in path planning compared with the DQN algorithm, and indicate the Double DQN algorithm can effectively process complex environmental information and make optimal path planning.