Game Theoretic Reinforcement Learning Framework For Industrial Internet of Things

The fifth-generation (5G) wireless net-work provides high-rate, ultra-low latency, and high-reliability connections that can meet the industrial IoT requirements in factory automation especially for swarm robotics communication. In this paper, we address 5G service provisioning in an automated warehouse scenario where swarm robotics is controlled by an industrial controller that provides routing and job instructions over the 5G network. Leveraging the co-ordinated multipoint (CoMP), we formulate a joint CoMP clustering and 5G ultra-reliable low-latency communication (URLLC) beamforming design problem to control the robots that move around the automated warehouse for goods storage with the planed reference tracks. Traditional iterative optimization approaches are impractical in such dynamic wireless environments due to high computational time. We propose a game-theoretic CoMP clustering algorithm combined with the Proximal Policy Optimization method to obtain a stationary solution closed to that of the exhaustive search algorithm considered as the global optimal solution.

I. Introduction

The "Fourth Industrial Revolution" is considered the automation revolution thanks to the innovations of 5G wireless communications, automation technologies, and artificial intelligence. Ultra-reliable and low-latency communication (URLLC) service provided by 5G wireless network is able to fulfill the stringent requirement of factory automation, e.g. 10⁻⁹ packet loss probability and 99.9999% availability in motion control and mobile robot use cases [1]. However, guaranteeing extremely high reliability is challenging in such a dynamic environment as an auto-mated warehouse with high mobility swarm robotics, i.e., automated guided vehicles (AGV). Coordinated Multi-Point (CoMP) communication technique [2] that lever-ages spatial diversity is promising to achieve URLLC by sending duplicate data streams over diverse paths. In the automated warehouse scenario, CoMP can combine the signal from multiple radio base stations (gNBs) so that highly dependable communications can be achieved to the moving objects, i.e., AGVs with the physical obstructions, e.g. warehouse racks and shelves.