Multiagent Reinforcement Learning-Based Semi-Persistent Scheduling Scheme in C-V2X Mode 4

The Third Generation Partnership Project has standardized cellular vehicle-to-everything (C-V2X) sidelink Mode 4 to support direct communication between vehicles. In Mode 4, the sensing-based semipersistent scheduling (SPS) scheme enables vehicles to autonomously reserve and select radio resources. In particular, SPS has three processes to realize the resource scheduling, including continuously sensing resources, probabilistically reselecting resources, and periodically reserving resources. However, vehicles randomly select resources from the available resource lists in the resource reselection process, resulting in frequent packet collisions especially when radio resources are insufficient. Unlike the traditional SPS, this paper proposes a multiagent deep reinforcement learning-based SPS (RL-SPS) algorithm to help vehicles select appropriate radio resources with the aim of reducing packet collisions. Furthermore, a multi-head attention mechanism is adopted to improve the training efficiency by helping vehicles selectively pay attention to the observations and actions of neighbouring vehicles. It is worth noting that the RL-SPS algorithm fits the characteristics of Mode 4, which selects resources without requiring any global information. Simulation results show that RL-SPS outperforms other decentralized approaches and demonstrate the scalability and robustness of RL-SPS in a dynamic vehicular network.

I. Introduction

With the rapid development of autonomous driving and networked vehicles, vehicle-to-everything (V2X) communication technology is regarded as one of the critical technologies for supporting the data exchange among vehicles and other network entities, such as edge servers and pedestrians [1]. There are two primary radio standards facilitating V2X communication: dedicated short-range communication (DSRC) [2] and cellular V2X (C-V2X) [3]. The former was built on IEEE 802.11p, and the latter was first introduced by the Third Generation Partnership Project (3GPP) in its Release 14 [4]. Supported by LTE and 5G technologies, C-V2X can provide broader coverage, better quality of service (QoS) guarantees, and a higher scalability than DSRC [5]. To support vehicle-to-vehicle (V2V) direct communications both in coverage and out of coverage of cellular networks, C-V2X introduces two modes for radio resource allocation: Mode 3 and Mode 4. As shown in Fig. 1, in Mode 3, radio resources are centrally allocated and scheduled by LTE base stations over the Uu interface. In Mode 4, vehicles autonomously select radio resources according to the sensing-based semipersistent scheduling (SPS) scheme via the PC5 interface. This paper focuses on the V2V communications based on the SPS scheme in C-V2X Mode 4.