Joint Rate and Power Optimization for Multimedia Streaming in Wireless Fading Channels via Parametric Policy Gradient

In this paper, we consider the two-timescale queueaware resource control for wireless video streaming applications. By applying a singular perturbation approach, we study a twotimescale joint optimization of transmission power and streaming rate control for real-time video streaming. The timescale parameter e is introduced to separate the state variables into slow and fast varying groups, and the power control is adaptive to both the channel state information and the playback queue state information (QSI) at the fast timescale, while the streaming rate representing the video quality is only adaptive to the QSI at the slow timescale. We first derive the averaged system by taking the perturbation parameter ε → 0, and the limiting problem is a firm approximation for the formal problem with small ε. Based on the averaged system, we derive the Hamilton-Jacobi-Bellman equation and show that the optimal control policy has a structural property which can further help us to simplify the problem. We use the parametric policy reinforcement learning algorithm to obtain the optimal parameterized policy, and show the algorithm convergence result. Finally, the proposed scheme's performance is compared with various baselines through simulations and it is shown that significant gain can be achieved.