Multi-Objective Optimization of AGV Real-Time Scheduling Based on Deep Reinforcement Learning

Driven by the trend away from mass production towards more customization and individualism, manufacturing is under massive price pressure. Therefore, continuously improving production efficiency and reducing costs have always been the focus of manufacturing companies. With recent advances in Industry 4.0 and industrial Artificial Intelligence (AI), Automated Guided Vehicles (AGVs) have become a very promising technology to support this trend. Nowadays, they are widely used in job shop environments for material handling. However, this promising technology goes along with various new challenges, such as the high dynamics, complexity, and uncertainty of the job shop environment for AGV scheduling. To address these challenges, an adaptive Deep Reinforcement Learning (DRL) based AGV real-time scheduling approach is approached to optimize several efficiency parameters of the overall job shop system. Firstly, the DRL optimization problem is formulated as Markov Decision Process (MDP). The state and action representation, reward, and optimal policy function are described in detail. Then a novel DRL method is further developed to achieve the optimal mixed rule policy. For that, a novel RL algorithm, Proximal Policy Optimization (PPO), is applied to the Deep Neural Network (DNN) and implemented in TensorForce, a reinforcement learning package in Python based on TensorFlow. This algorithm is compared to conventional heuristic rules. Furthermore, SimPy, a relatively new discrete-event simulation package in Python, is used to implement the job shop environment. The job shop environment is based on a real-world scenario of a semiconductor factory, which is implemented in a simplified manner and applied to the DRL agent. The results are presented afterward, followed by a feasibility and effectiveness analysis of this approach.