Formation Control Optimization via Virtual Leader Exploration with Deep Reinforcement Learning for Unmanned Aerial Vehicles

Compared to single unmanned aerial vehicle (UAV), multi-UAVs formation has garnered significant attention due to their advantages in collaborative task execution, task allocation, as well as heightened redundancy and reliability. This paper investigates a UAV rendezvous system comprising multiple UAVs with random positions and velocities. We introduce the concept of a virtual leader, with the objective of transforming the formation control problem of UAVs into tracking the positional movements of the virtual leader. We model the exploration of the optimal position for the virtual leader as a Markov decision process and propose a variablestep exploration algorithm combined with deep reinforcement learning techniques, guiding the virtual leader towards its optimal position from its initial location. The experimental simulation results ultimately demonstrate the effectiveness of our approach.