Priorformer: A UGC-VQA Method With Content and Distortion Priors

User Generated Content (UGC) videos are susceptible to complicated and variant degradations and contents, which prevents the existing blind video quality assessment (BVQA) models from good performance since the lack of the adaptability of distortions and contents. To mitigate this, we propose a novel prior-augmented perceptual vision transformer (PriorFormer) for the BVQA of UGC, which boots its adaptability and representation capability for divergent contents and distortions. Concretely, we introduce two powerful priors, i.e., the content and distortion priors, by extracting the content and distortion embeddings from two pre-trained feature extractors. Then we adopt these two powerful embeddings as the adaptive prior tokens, which are transferred to the vision transformer backbone jointly with implicit quality features. Based on the above strategy, the proposed PriorFormer achieves state-of-the-art performance on three public UGC VQA datasets including KoNViD-1K, LIVE-VQC and YouTube-UGC.

1. Introduction

Millions of User Generated Content (UGC) videos have sprung up with the rapid development of multi-media and mobile camera technologies, which brings a crucial and promising challenge, i.e., how to measure the subjective quality of UGC videos accurately and properly. Different from other video datasets, UGC videos are usually acquisited and uploaded by amateur photographers. Therefore, the UGC videos are susceptible to extremely diverse and complicated degradations, i.e., the hybrid distortions, including underexposure, overexposure, jitter, noise, color shift, etc. Apart from this, the content of UGC videos is generally very diverse due to the low requirements for shooting locations, such as natural scenes, animations, games, screen content, etc. These two aspects (complicated distortions and diverse contents) severely hinder the application of existing video quality assessment (VQA) methods to UGC videos. It is urgent to investigate an effective UGC VQA method to overcome this challenge and achieve human-like quality assessment for UGC videos