Hierarchical Path Planning based on PPO for UVs on 3D Off-Road Terrain

In 3D off-road terrain, the driving of the unmanned vehicle (UV) is influenced by the combined effect of terrain and obstacles, leading to greater challenges in safety and stability. Many path planning studies on 3D off-road terrain are lacking in the consideration of this combined effect, for which we implemented a continuous simulation system and made the following contributions. First, an integrated potential field that is adaptive to the size of obstacles is proposed, and then it is used as a movement cost in the Voronoi diagram (VD) to obtain a pre-planned path for obstacle avoidance. Second, the Proximal Policy Optimization (PPO) algorithm is used to reduce the task objective, the effects of terrain and obstacles, and vehicle motion constraints to an objective function to be optimized. Finally, to improve the learning efficiency and generalization ability, we propose a hierarchical path planning (HPP) method to introduce the derived pre-planned path into the learning process of the PPO algorithm. It is experimentally demonstrated that the PPO algorithm combined with the HPP method is able to accomplish the path planning task in 3D off-road terrain of different sizes and difficulties, and obtains higher accuracy and shorter 3D path than the shaping reward (SR) method. The final path is suitable for the UV without smoothing, and the UV automatically adjusts its steering on undulating terrain to improve stability, and automatically slows down to pass through sharp turns to reduce the risk of rollover.