Mobility-Aware AR Computation Tasks Offloading Based on Offline RL over Terahertz System

Augmented reality (AR) technology is increasingly applied in the Internet of vehicles (IoV). As it poses significant challenges to the communication and computation capabilities of AR terminal devices, this paper investigates combining terahertz (THz) and mobile edge computing (MEC) integrated with reinforcement learning (RL) to optimize the performance of AR services for the IoV. Though the RL-based MEC offloading has been validated to have excellent performance, it has an intrinsic disadvantage where its training phase needs inappropriate offloading strategies to be tested frequently and poses risks to safe driving. Moreover, although THz communication can provide high transmission data rates, its characteristics that different transmission distances lead to different available bandwidth, pose a unique challenge for the communication resources allocation. Motivated by these, we propose a novel static data-driven AR tasks offloading framework to solve resources allocation problem for the IoV with THz communication. To deal with the challenge that different users have different available THz bandwidth, we propose a distance-aware fair THz bandwidth allocation algorithm. Moreover, the offline RL, instead of online RL, is utilized to train the AR task offloading decision agent, where interaction with the real world is not required, completely safeguarding the system's safety. To address the typical issue of out-of-distribution (ODD) actions encountered in offline RL, we adopt conservative Q-learning (CQL) to train and obtain the optimal AR computation task offloading strategy to enhance users' experience. Simulation results show that the proposed solution outperforms state-of-the-art algorithms in terms of algorithm stability and user reward.