Video Quality Assessment of User Generated Content: A Benchmark Study and a New Model

Recent years have witnessed an explosion of user-generated content (UGC) shared and streamed over the Internet. Accordingly, there is a great need for accurate video quality assessment (VQA) models for consumer or UGC videos to monitor, control, and optimize this vast content. Here we contribute to advancing the UGC-VQA problem by conducting a comprehensive evaluation of leading blind VQA (BVQA) models. Besides, we also created a new fusion-based BVQA model, which we dub the VIDeo quality EVALuator (VIDEVAL), that effectively balances the trade-off between performance and efficiency. Our experimental results show that VIDEVAL achieves state-of-the-art performance at a lower computational cost. We believe our reliable and reproducible benchmark will facilitate further research on deep learning-based BVQA modeling. An implementation of VIDEVAL has been made available online ¹.¹https://github.com/vztu/VIDEVAL_release

1. Introduction

Recently, there has been tremendous growth in social media, where huge volumes of user-generated content (UGC) is shared over media platforms such as YouTube, Facebook, and TikTok. Indeed, the prevalence of UGC has started to shift the focus of video quality research from legacy synthetically-distorted databases to newer, larger-scale authentic UGC datasets, which are being used to create solutions to what we call the UGC-VQA problem. UGC-VQA studies typically follow a new design paradigm whereby: 1) All the source content is consumer-generated instead of professional-grade, thus suffers from unknown and highly diverse impairments; 2) they are only suitable for developing no-reference models, since reference videos are unavailable; 3) the types of distortions are authentic and commonly intermixed, and include but are not limited to in-capture, editing and processing artifacts, compression and transcoding distortions. Moreover, compression artifacts are not necessarily the dominant factors affecting video quality, unlike legacy VQA datasets and algorithms. These unpredictable perceptual degradations make quality prediction of UGC consumer videos very challenging.