Maximum Entropy and Quantized Metric Models for Absolute Category Ratings

The datasets of most image quality assessment studies contain ratings on a categorical scale with five levels, from bad (1) to excellent (5). For each stimulus, the number of ratings from 1 to 5 is summarized and given in the form of the mean opinion score. In this study, we investigate families of multinomial probability distributions parameterized by mean and variance that are used to fit the empirical rating distributions. To this end, we consider quantized metric models based on continuous distributions that model perceived stimulus quality on a latent scale. The probabilities for the rating categories are determined by quantizing the corresponding random variables using threshold values. Furthermore, we introduce a novel discrete maximum entropy distribution for a given mean and variance. We compare the performance of these models and the state of the art given by the generalized score distribution for two large data sets, KonIQ-10k and VQEG HDTV. Given an input distribution of ratings, our fitted two-parameter models predict unseen ratings better than the empirical distribution. In contrast to empirical distributions of absolute category ratings and their discrete models, our continuous models can provide fine-grained estimates of quantiles of quality of experience that are relevant to service providers to satisfy a certain fraction of the user population.

I. Introduction

In General, image processing applications with the goal of outputting images for human consumption can benefit from models of perceived image quality. Often, such models are created by machine learning, which typically requires training sets of images with annotations of perceived quality. To this end, human subjects are presented with images and asked to rate their visual quality, usually on an absolute category rating (ACR) scale, i.e., they are asked to select a quality from the scale . However, subjects might disagree, and so many ratings are collected for each image. The collected distribution of ratings for an image is summarized by the mean opinion score (MOS), i.e., the average rating [1].