Dynamic Scheduling of Gantry Robots using Simulation and Reinforcement Learning

Industry 4.0 induces an increasing demand of autonomous interaction between the units of production facilities, like work centers and transportation equipment. This has an impact on the requirements for production scheduling and control algorithms. These must be capable to adapt autonomously to changes on the shop floor. This paper presents a combination of Reinforcement Learning and discrete event simulation for controlling a flexible flow shop using a gantry robot system as transportation unit. In a gantry robot system, parts are transported by carriages fitted with grippers that travel along rails from machine to machine. The presented RL-agent learns autonomously the right control policy to move the carriages. It is shown that in cases the optimal policy can be determined the Reinforcement Learning based policy is optimal and in other cases the achieved throughput does slightly exceed the throughput gained by a heuristic priority rule for controlling the gantry robot.