Delayed Soft Actor-Critic Based Path Planning Method for UAV in Dense Obstacles Environment

In order to improve the convergence performance of soft actor-critic (SAC) algorithm in path planning problems, a delayed prioritized experience replay soft actor critic (DPERSAC) is proposed by designing a novel experience replay mechanism in a non-uniform manner for decreasing the convergence time. The path planning mathematical model is built for unmanned aerial vehicles (UAVs) subject to the flight performance constraints and obstacle avoidance constraints. Then the three typical elements of SAC are customized to satisfy the requirements of UAV's path planning. Differ from the traditional update manner that the soft Q-function network and policy network are updated recursively, the soft Q-function network is updated conditionally firstly and the policy network is subsequently iterated based on the trained soft Q-function in this paper. Finally, the Monte Carlo simulation results demonstrate that the computational time of the proposed DPERSAC method is only 4% of the rolling-based sparse A * algorithm in the dense obstacle environment.