An Intelligent Multi-Layer Control Architecture for Logistics Operations of Autonomous Vehicles in Manufacturing Systems

In this paper, autonomous vehicles are considered for addressing logistic operations in manufacturing systems. The starting idea consists in organizing a given group of autonomous robots/vehicles in a finite set of platoons in charge to accomplish prescribed job(s) within the manufacturing system. Three aspects are then needed to be formally outlined: task scheduling, routing decisions and command inputs computations. Here, a new distributed multi-layer architecture has been conceived by using three methodologies: timed colored Petri nets, deep reinforcement learning and model predictive control. Roughly speaking, timed colored Petri nets are exploited to formally model the manufacturing system so that an optimal scheduling task complying with the required jobs and the available vehicles is derived; then, run-time routing decisions are obtained by using a distributed reinforcement learning algorithm which exploits the available information provided by the vehicle sensor module; finally, the distributed model predictive control algorithm is built by resorting to a set-theoretic approach where most of the computations are off-line performed. A flexible manufacturing system consisting of four machines and a Load/Unload station is used for simulation purposes. Specifically, five jobs are hypothesized and some scenarios with an increasing number of available vehicles are simulated. In order to evaluate the benefits of the proposed approach, a time criterion based on the completion of all the jobs is considered with the aim to put in light that increasing the number of vehicles improves the control performance until congestion phenomena become unavoidable.
Note to Practitioners
—This paper proposes an innovative methodology for addressing the logistic operations within flexible manufacturing systems (FMSs) by using a team of autonomous ground vehicles. Unlike existing approaches, the core of this framework consists in combining along a hierarchical structure the capabilities of timed colored Petri nets and the deep reinforcement learning techniques to determine a near-optimal scheduling and run-time routing decisions that are provided to the distributed model predictive units in charge to accomplish the prescribed task. This multi-layer architecture has two main merits: a single platoon, completely disconnected from the others, is devoted to perform its own job; computational burdens are affordable during the on-line operations because most of the computations are moved in the off-line phase.