A Quality-of-Experience-Aware Framework for Versatile Video Coding-Based Video Transmission

Multimedia applications have expanded and resulted in huge demands for high resolution videos in order to enhance the end user Quality of Experience (QoE). Therefore, QoE becomes a critical factor that is often addressed in design considerations of Consumer Electronics (CE) devices. Furthermore, error prone transmission conditions can influence the QoE, potentially degrading the end-user multimedia experience. Although video coding standards such as Versatile Video Coding (VVC) take potential mitigation measures to improve the bandwidth reductions, it fails to address other significant network Quality of Service (QoS) parameters that impact transmission based QoE such as transmission delay and Packet Loss Rate (PLR) in the encoding processes. In response, this paper first derives a relationship between QoE that addresses the network QoS parameters and VVC dependent parameters. Subsequently, the derived QoE model is used with the VVC encoding process to adaptively optimize Lagrange parameters as well as the quantization parameter. Extensive experiments report significant improvements to VVC Test Model (VTM) 7.0, achieving superior QoE gains over the state-of-the-art research and up to 3.41% over the anchor implementations.

I. Introduction

The estimates project that video applications make up 69% of the world’s mobile data traffic, with an increase to about 79% expected in 2027 accounting for the growing popularity of video applications [1], highlighting the importance of visual media. Consumer Electronics (CE) devices have started to penetrate these new markets and consequently demand critical design considerations during media capturing, processing, representation, and visualization [2], [3]. Furthermore, the increasing popularity of applications such as Virtual reality (VR), high dynamic range video (HDR), and video formats such as 2K, 4K, and 8K in various CE devices provide dynamics in video applications with the expectation to meet the user’s quality of experience (QoE). Hence, these high-resolution videos bring need to be optimally compressed to give rise to high video quality playback for end-users [4]. To this end, incorporating QoE information in the optimization procedures of the existing video compression infrastructures can enhance multimedia consumption in CE devices.