Multi-agent Deep Reinforcement Learning Based Integrated Scheduling of Machines and AGVs in Discrete Manufacturing Workshop

Discrete manufacturing workshops (DMW) are effective tools for modern manufacturing enterprises to improve productivity and reduce costs. Highly flexible Automated Guided Vehicles (AGVs) are also essential transportation devices in workshops. Therefore, a multi-agent deep reinforcement learning (DRL) algorithm framework is proposed for the integrated scheduling problem of machines and AGVs (ISPMA) in DMW with the goal of minimizing the maximum order completion time. Firstly, a new state space is designed to comprehensively describe the actual production situation of the workshops. Secondly, universal rules for selecting machines, AGVs, and workpiece sorting are designed to improve the generalization of scheduling decisions. Finally, a multi-agent Proximal Policy Optimization with a global critic (MAPPO-GC) is designed for the cooperation and interaction of multi-agent systems. and it was trained offline and applied online to quickly respond to ISPMA. Examples verify that the MAPPO-GC proposed in this paper has excellent learning performance and can effectively solve ISPMA in DMW.