Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

Reconfigurable intelligent surface (RIS) is a pivotal technology in communication, offering an alternative path that significantly enhances the link quality in wireless communication environments. In this paper, we propose a RIS-assisted internet of vehicles (IoV) network and consider the vehicle-to-everything (V2X) communication method. In order to improve the timeliness of vehicle-to-infrastructure (V2I) links and the stability of vehicle-to-vehicle (V2V) links, we introduce the age of information (AoI) and the payload transmission probability models. Thus, we aim to minimize the AoI of V2I links and prioritize transmission of V2V links payload. In this framework, the base station (BS) server acts as the agent responsible for resource allocation for vehicles and controlling the phase-shift of the RIS. We use the Soft Actor-Critic (SAC) algorithm to address this problem due to its gradual convergence and high stability in the optimization process. An AoI-aware joint vehicular resource allocation and RIS phase-shift control scheme based on SAC algorithm is proposed and simulation results show that its convergence speed, cumulative reward, AoI performance, and payload transmission probability outperform those of proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3) and stochastic algorithms.

I. Introduction

In Recent years, with the rapid advancement of Internet of Things (IoT), vehicles have undergone a significant transformation towards increased intelligence [1], [2]. This has led to a higher demand for vehicle communication technology, prompting numerous organizations to engage in in-depth research to meet diverse requirements in internet of vehicles (IoV) networks [3], [4], [5], [6]. Vehicle-to-everything communication (V2X), as an important technology, has brought multiple enhancements to vehicles in communication [7], [8]. V2X technology covers a variety of scenarios, including vehicle-to-vehicle communication (V2V), vehicle-to-infrastructure communication (V2I), vehicle-to-pedestrian communication (V2P), and vehicle-to-network communication (V2N) [9], [10]. This diversity of communication scenarios provide rich possibilities for different applications. For example, V2I communication can enable vehicles to obtain road conditions, traffic signal conditions, driving routes from base station (BS), V2V communication enables vehicles to exchange key information such as position, speed, and acceleration with each other [11], [12], [13].