Deep Q-network Based UAV Autonomous Obstacle Avoidance with Prior Reward Shaping

Autonomous decision-making methods in complex environments have become a crucial technology for unmanned aerial vehicles (UAVs) to expand their application scenarios. In this paper, a UAV obstacle avoidance method based on Deep Q-Network (DQN) is present to achieve end-to-end UAV sensor-perception-to-action mapping. In this study, we form the real-time perceived observation of the UAV based on the target location, UAV orientation information, and the distance to the obstacle sensed by the ray sensor, and generate the corresponding decision by evaluating the current state. To improve the convergence of the approach, we consider the external factors that affect the UAV policy and design a set of reward functions based on prior knowledge which suitable for autonomous UAV obstacle avoidance. With reward function design, the UAV can be guided to take more accurate actions, enabling efficient training and safe navigation. To validate the performance of the algorithm, a typical UAV obstacle avoidance scenario in the Unity3D environment is built and carried out algorithm evaluation. Experimental results show that the reward function can support the UAV to achieve convergence of the policy with limited training steps, which is a significant advantage to the comparison methods.