Multi-Agent Formation Control With Obstacle Avoidance Using Proximal Policy Optimization

In this paper, a formation of second-order holonomic agents is made to navigate through an obstacle field using proximal policy optimization (PPO) based deep reinforcement learning (DRL). The angle-based formation is allowed to shrink while maintaining its shape in order to navigate through tight spaces and take the geometric centroid of the formation towards the goal. Two reward schemes are presented, one based on the actions of individual agents and another based on the actions of the team as a whole. For each case, all the agents share a single policy that is trained in a centralized manner. Distance measurements, state information, error information regarding neighboring agents, and simulation information are used for training each policy in an end-to-end fashion. Simulation results for both approaches are compared.