QoE-based Deep Reinforcement Learning for Resource Allocation in Real Time XR Video Transmission

With the rapid development of mobile internet technology, extended reality (XR) applications have significantly increased. However, the ultra low latency and ultra high speed requirements of XR pose a huge challenge to wireless resource allocation. Therefore, this paper studies the dynamic resource block (RB) allocation for real-time cloud XR video transmission. Unlike existing works focusing on static RB allocation, we consider a dynamic assignment strategy appropriated for a user-moving scenario and employ deep reinforcement learning (DRL) methods. Specifically, we propose and compare two DRL approaches based on the soft actor-critic (SAC) model with powerful exploration capabilities: a centralized single-SAC (S-SAC) and a distributed multi-SAC (M-SAC) method. The former treats the base station as a single agent, while the latter treats all RBs as different independent agents. Multi-agent model reduces the neural network’s exponential output dimension to a linear order of magnitude. Moreover, we develop a novel reward function combining immediate and delayed rewards to solve the reward sparsity problem in DRL. Extensive simulations demonstrate that the proposed methods obtain more than 50% performance gain over various benchmarks for both fixed and random video frame size scenarios.