Formation and Collision Avoidance via Multi-Agent Deep Reinforcement Learning

Formation with collision avoidance is the fundamental problem in multi-agent systems. Traditional control methods suffer from some challenges such as relying on global information and requiring rigid adherence to predefined rules. Such limitations result in poor performance of traditional solutions in complex environments. To overcome such drawbacks, this paper proposes a multi-agent proximal policy optimization-based formation and collision avoidance approach without using global information. A model suitable for complex environments is designed, alongside a formation control strategy based on relative information. Subsequently, relative formation error is incorporated into the reward function to enhance the performance and stability of the formation strategy. Finally, a simulation example is given to verify the validity of the theoretical results.

1 Introduction

Multi-agent system has garnered increasing attention from various field, including traffic control [1], [2], emergency res-cue missions [3]–[5], and industrial automation [6], [7]. Formation control with obstacle avoidance is one of the key techniques of the MAS, ensuring agents to navigate and explore environments with high efficiently. The conventional multi-agent formation control methods are chiefly categorized into leader-follower and behavior-based.