An Intelligent MAV-UAV Cooperative Combat Planning Method Based on Deep Reinforcement Learning

With the development of intelligent airforce combat, deep reinforcement learning (DRL) has been widely applied in various combat systems. Among these systems, the hybrid system of the manned aerial vehicle (MAV) and unmanned aerial vehicle (UAV) with cooperative combat mode shows bright foreground. However, the heterogeneity and dynamics in the formation pose challenges to the cooperative combat planning algorithm. To this end, this paper proposes an intelligent MAV-UAV cooperative combat planning method based on discretized proximal policy optimization (PPO). A 5×5 rule-based action space is constructed based on the knowledge of air combat. The state space and the corresponding reward function are introduced to stably train the policy. In contrast to other continuous DRL methods and multi-agent DRL, the proposed method is easier to be reproduced and be trained, and the defined rule-based formation actions can be expanded to suit other scenes. The feasibility and effectiveness of the proposed planning method are validated by numerical experiments based on the XSimStudio.