Assessing the Performance of Image Quality Assessment Metrics

Quality assessment of tone-mapped images has been an active research area over the past years. Humans have a personal liking of image attributes such as vividness of colors, mean brightness level, and contrast. Therefore, image quality is often determined through subjective studies. The participants are asked to assign scores to the images, and the average scores are used to determine the relative quality of the images or the algorithms that produced them. Several metrics have been proposed that try to replicate this process and assign quantitative scores to the images. While these metrics perform reasonably in general and their scores correlate well with subjective scores, they can fail badly in some situations. This paper proposes a novel method to put these metrics to test under extreme conditions and observe their performance. For this, we use the differential evolution approach that keeps modifying the tone-mapping function iteratively, such that the assigned score by the metric keeps increasing over iterations. We show that visible distortions start showing up in the image in many situations, yet the score given by the metric remains very high.