Reinforcement Learning for Underwater Spatiotemporal Path Planning, with Application to an Autonomous Marine Current Turbine

This paper presents a reinforcement learning (RL) framework applied for an autonomous underwater vehicle (AUV) path planning, focusing on a specific type of energyharvesting AUV, entitled marine current turbine (MCT). The proposed RL-based approach improves a classical path planning to adopt with an underwater environment prone to spatiotemporal uncertainties. The path planning problem is formulated to achieve the goal of maximizing the harnessed energy from the MCT subject to the agent dynamics and the spatiotemporal environment constraints. Three RL algorithms, including Q-learning, deep Q-network (DQN), and proximal policy optimization (PPO), are nominated to deal with the path planning over both discrete gridded and continuous underwater environments modeling. The experimental results demonstrate the efficiency of the RL-based approaches in seeking the optimal path in the underwater environment, where further discussion is presented to generalize the proposed approach to other energy-harvesting autonomous vehicles operating in the spatiotemporally varying environment, such as airborne wind turbines.