Blind Image Quality Index for Authentic Distortions With Local and Global Deep Feature Aggregation

Blind image quality assessment (BIQA) for authentic distortions is still a great challenge, even in today’s deep learning era. It has been widely acknowledged that local and global features are both indispensable for IQA, which play complementary roles. While combining local and global features is straightforward in traditional handcrafted feature-based IQA metrics, it is not an easy task in the deep learning framework. This is mainly due to the fact that deep neural networks typically require input images with a fixed size. Current metrics either resize the image or use local patches as input, which are problematic in that they cannot integrate local and global aspects as well as their interactions to achieve comprehensive quality evaluation. Motivated by the above facts, this paper presents a new BIQA metric for authentic distortions by aggregating local and global deep features in a Vision-Transformer framework. In the proposed metric, selective local regions and global content are simultaneously input for complementary feature extraction, and the Vision-Transformer is employed to build the relationship between different local patches and image quality. Self-attention mechanism is further adopted to explore the interaction between local and global deep features, producing the final image quality score. Extensive experiments on five authentically distorted IQA databases demonstrate that the proposed metric outperforms the state-of-the-arts in terms of both prediction performance and generalization ability.

I. Introduction

Image quality assessment (IQA) is a long-standing challenge in computer vision, and it is vital to many image processing problems [1]–[11], including image acquisition, compression, enhancement, generation, and retrieval. In the past decades, a great number of IQA metrics have been proposed [12]–[27], which can be divided into full-reference (FR), reduced-reference (RR), and no-reference (NR) [1], [5]. Since reference images are not needed, NR-IQA, or called blind IQA (BIQA), has the widest applications in real-world scenarios. With the development of deep learning, nowaday’s BIQA metrics are putting more emphasis on authentic distortions. Although significant advances have been achieved, deep BIQA metrics are far from ideal in terms of both prediction accuracy and generalization ability.