RAM360: Robust Adaptive Multi-Layer 360$^\circ$ Video Streaming With Lyapunov Optimization

Viewport-adaptive streaming approaches are emerging as the most promising way to deliver high-quality 360${^\circ }$ videos. The viewport prediction techniques are developed to reduce bandwidth waste and improve users’ Quality of Experience (QoE). However, the viewport prediction result is only reliable with a short prediction window, i.e., a short playback buffer, which conflicts with maintaining a long buffer to minimize the stall ratio. To deal with this problem, we present RAM360, a Robust Adaptive Multi-layer 360${^\circ }$ video streaming system, to ensure high viewport quality and low stall ratio concurrently. We make three technical contributions. First, we design a QoE-driven robust multi-layer streaming framework, where each chunk is encoded into multiple independent layers with different quality levels. The client can dynamically decide which chunk and which layer to download according to their QoE contributions. Thus, the client can enhance the low-quality chunks (including the mistakenly predicted ones) in time to improve the viewport quality. Meanwhile, the client can adaptively download new chunks to the buffer to decrease the risk of stall. Second, we establish a novel model as users’ QoE metric throughout the playback progress, aiming to guide the client’s download theoretically. Third, we utilize the Lyapunov optimization theory to solve the QoE optimization problem online while assuring our algorithm’s near-optimality. We demonstrate that RAM360 can significantly outperform the existing schemes regarding the QoE (related to viewport quality, viewport quality oscillation, and stall ratio) through extensive experiments on public datasets.

I. Introduction

The latest development of 5G networks and specialized hardware codecs enable 360 video applications to become popular at a staggering speed. Compared with conventional videos, 360 videos are much larger (4× to 6×) due to their panoramic nature. Nevertheless, users can only see the viewport of the 360 videos at every moment. Therefore, delivering the entire 360 video panorama with the same quality brings an inevitable waste of bandwidth and computational resources. Besides, since the motion-to-photon latency requirement of 360 videos (typically 5 41 ms [1]) is much smaller than the Internet request-reply delay (typically 50 400 ms [2]), the client must prefetch the video content users will see in the future to avoid rendering a blank screen. Hence, tile-based viewport-adaptive streaming approaches are proposed [3]. They predict the user’s future viewport and prefetch high-bitrate tiles to this area and low-bitrate tiles to other regions to guarantee the viewport quality while reducing bandwidth waste.