Data-Driven Video Scene Importance Estimation for Adaptive Video Streaming

Recently, many video streaming services have adopted adaptive bitrate algorithms as their optimization algorithm. Traditionally, ABR algorithms strive to provide an accurate estimate of network conditions. In recent years, ABR algorithms have incorporated the content of interest of the video into the algorithm based on the fact that users are interested in certain segments of the video when watching, and they have achieved significant performance improvements. However, these efforts are expensive in terms of time and cost and are difficult to adapt to new videos. To overcome these limitations, we propose a system for estimating scene saliency for new videos. To do so, we first build a dataset from a large-scale video streaming service, which is then trained on a deep learning model consisting of a 3D CNN and a Transformer. As a result, our proposed model achieves a significantly lower prediction error rate on unseen videos and achieves a significant QoE improvement when incorporated with the ABR algorithm.