Real-time Guidance for Powered Landing of Reusable Rockets via Deep Reinforcement Learning

Powered landing of reusable rocket is an advanced technology to achieve pinpoint landing (the norm of position error < 5 m and velocity error < 2 m/s) while satisfying a series of highly nonlinear constraints. A major challenge is guaranteeing fuel-optimal and convergence when solving rocket powered landing problem. In this manuscript, a real-time feedback guidance algorithm based on deep reinforcement learning is developed. The proposed method maps state directly to thrust control commands. The first contribution of this paper is to use multi-stage reward function to eliminate the negative effects triggered by design guidance law, thereby significantly enhancing fuel-optimal performance. Another contribution is that a model pre-training framework based on imitation learning is presented to improve model convergence by fitting optimal data. Numerical experiments show that the nearly fuel-optimal trajectories generated by the proposed algorithm successfully achieve pinpoint landing from random initial states.