Game of Drones: Multi-UAV Pursuit-Evasion Game With Online Motion Planning by Deep Reinforcement Learning

As one of the tiniest flying objects, unmanned aerial vehicles (UAVs) are often expanded as the “swarm” to execute missions. In this article, we investigate the multiquadcopter and target pursuit-evasion game in the obstacles environment. For high-quality simulation of the urban environment, we propose the pursuit-evasion scenario (PES) framework to create the environment with a physics engine, which enables quadcopter agents to take actions and interact with the environment. On this basis, we construct multiagent coronal bidirectionally coordinated with target prediction network (CBC-TP Net) with a vectorized extension of multiagent deep deterministic policy gradient (MADDPG) formulation to ensure the effectiveness of the damaged “swarm” system in pursuit-evasion mission. Unlike traditional reinforcement learning, we design a target prediction network (TP Net) innovatively in the common framework to imitate the way of human thinking: situation prediction is always before decision-making. The experiments of the pursuit-evasion game are conducted to verify the state-of-the-art performance of the proposed strategy, both in the normal and antidamaged situations.

I. Introduction

Competition and cooperation are the two essential elements for lots of complex ecosystems in nature, just like a pack of wolves hunting the sheep on the prairie. They are deemed as the driving forces of biological and social evolution. With the characteristics of lightweight, high maneuverability, and low cost, it is hard for a single quadcopter to accomplish complex missions alone. Therefore, the quadcopters have been derived “unmanned swarm system” application mode, which contains cooperation among partners and competition with enemies.