Global-and-Local Attention-Based Reinforcement Learning for Cooperative Behaviour Control of Multiple UAVs

Due to the strong adaptability and high flexibility, unmanned aerial vehicles (UAVs) have been extensively studied and widely applied in both civil and military applications. Although UAVs can achieve significant cost reduction and performance enhancement in large-scale systems by taking full advantage of their cooperation and coordination, they result in a serious cooperative behaviour control problem. Especially in dynamic environments, the cooperative behaviour control problem which has to quickly produce a safe and effective behaviour decision for each UAV to achieve group missions, is NP-hard and difficult to settle. In this work, we design a global-and-local attention-based reinforcement learning algorithm for the cooperative behaviour control problem of UAVs. First, with the motion and coordination models, we analyze the collision avoidance, motion state update, and task execution constraints of multiple UAVs, and abstract the cooperative behaviour control problem as a multi-constraint decision-making one. Then, inspired from the human-learning process where more attention is devoted to the important parts of data, we design a multi-agent reinforcement learning algorithm with a global-and-local attention mechanism to cooperatively control the behaviours of UAVs and achieve the coordination. Simulation experiments in a multi-agent particle environment provided by OpenAI are conducted to verify the effectiveness and efficiency of the proposed approach. Compared with baselines, our approach shows significant advantages in mean reward, training time, and coordination effect.

I. Introduction

In Recent decades, unmanned aerial vehicles (UAVs) have attracted tremendous research interest and are widely adopted in both civil and military applications due to their strong adaptability and high flexibility [1]. Compared with manned vehicles, UAVs free people from dangerous, boring, and burdensome works and carry out these works more efficiently [2]. As an important part of the new strategic industries, UAV systems featured by high automation and advanced technology have been emerging as a core competitiveness for most countries and regions worldwide to achieve industrial upgrading and technological innovation.