Path Planning Based on Deep Reinforcement Learning for Autonomous Underwater Vehicles Under Ocean Current Disturbance

The path planning issue of the underactuated autonomous underwater vehicle (AUV) under ocean current disturbance is studied in this paper. In order to improve the AUV’s path planning capability in the unknown environments, a deep reinforcement learning (DRL) path planning method based on double deep Q Network (DDQN) is proposed. It is created from an improved convolutional neural network, which has two input layers to adapt to the processing of high-dimensional environments. Considering the maneuverability of underactuated AUV under current disturbance, especially, the issue of ocean current disturbance under unknown environments, a dynamic and composite reward function is developed to enable the AUV to reach the destination with obstacle avoidance. Finally, the path planning ability of the proposed method in the unknown environments is validated by simulation analysis and comparison studies.

I. Introduction

With the development of marine science and the advancement of artificial intelligence and intelligent systems, the autonomous underwater vehicles (AUVs) are evolving towards the self-learning and adaptive [1]. At present, the most of AUVs used for deep-water exploration are underactuated AUVs [2]. It’s only includes stern thruster generally, and steering and pitching are realized through vector propulsion or rudder [3], [4]. Path planning is one of the core problems in the underactuated AUV fields. Its purpose is to find an optimal path from the beginning to the end [5], [6]. The path planning environment is either static or dynamic. In a static environment, the global environmental information such as terrain, obstacles and disturbances is known and a path can be planned ahead of the detection. However, for the dynamic environments, the global environmental information is unknown and the path needs to be planned in real-time [7]. Relatively speaking, the real time path planning in the dynamic environments has more practical significance and great difficulty.