Actor-Critic Based Back-off Algorithm for Massive Machine-Type Communication

Massive machine-type communication (mMTC) is one of the important technologies proposed for 5G and next-generation mobile communications, providing wireless communication services for an increasing variety of machine-type communication devices. Random access collision is one of the key problems in the mMTC scenario based on the cellular network architecture. To improve the probability of successful access, virtual small cells are first dynamically divided according to geographic location and application demand activity, and then machine-type communication devices are classified according to different requirements of the quality of service (QoS). Based on this system model, a back-off algorithm is proposed based on a deep reinforcement learning algorithm. Combined with the state of preambles and devices, the optimal back-off access strategy is obtained using analytic hierarchy process and Actor-Critic methods. The simulation results show that this method can reduce the collisions of back-off access process effectively and reduce the average delay in the system while meeting the demands of the overall QoS.

I. Introduction

Massive machine-type communication (mMTC) is one of the three core scenarios of the fifth generation mobile communication system (5G) and beyond 5G (B5G) [1], which is designed to support connectivity to a million-level density per square kilometer of machine-type device. It supports a large number of devices to perform grant-free random access (GFRA) and is widely applied in fields including the industrial InterNet, the InterNet of Things (loTs), vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V21) communication, and so on. The mMTC has the characteristics of small equipment size, low transmission frequency, diversified business, and a massive number of devices, which brings new challenges to device access.