A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment

In the field of objective image quality assessment (IQA), Spearman's ρ and Kendall's τ, which straightforwardly assign uniform weights to all quality levels and assume that each pair of images is sortable, are the two most popular rank correlation indicators. These indicators can successfully measure the average accuracy of an IQA metric for ranking multiple processed images. However, two important perceptual properties are ignored. First, the sorting accuracy (SA) of high-quality images is usually more important than that of poorquality images in many real-world applications, where only top-ranked images are pushed to the users. Second, due to the subjective uncertainty in making judgments, two perceptually similar images are usually barely sortable, and their ranks do not contribute to the evaluation of an IQA metric. To more accurately compare different IQA algorithms, in this paper, we explore a perceptually weighted rank correlation indicator, which rewards the capability of correctly ranking high-quality images and suppresses the attention toward insensitive rank mistakes. Specifically, we focus on activating a “valid” pairwise comparison of images whose quality difference exceeds a given sensory threshold (ST). Meanwhile, each image pair is assigned a unique weight that is determined by both the quality level and rank deviation. By modifying the perception threshold, we can illustrate the sorting accuracy with a sophisticated SA-ST curve rather than a single rank correlation coefficient. The proposed indicator offers new insight into interpreting visual perception behavior. Furthermore, the applicability of our indicator is validated for recommending robust IQA metrics for both degraded and enhanced image data.

I. Introduction

In most image processing applications, human eyes are the ultimate receiver of the visual signal, and a subjective test could offer ideal evaluations for various process results. However, confronted with the massive image data generated in our daily life, it has become impossible to manually annotate image quality via substantial human subjects, which is usually a time-consuming, troublesome and expensive process [1]. As an alternative, it is more economical and practical to implement well-designed objective image quality assessment (IQA) metrics.