Blind Stereoscopic Video Quality Assessment: From Depth Perception to Overall Experience

Stereoscopic video quality assessment (SVQA) is a challenging problem. It has not been well investigated on how to measure depth perception quality independently under different distortion categories and degrees, especially exploit the depth perception to assist the overall quality assessment of 3D videos. In this paper, we propose a new depth perception quality metric (DPQM) and verify that it outperforms existing metrics on our published 3D video extension of High Efficiency Video Coding (3D-HEVC) video database. Furthermore, we validate its effectiveness by applying the crucial part of the DPQM to a novel blind stereoscopic video quality evaluator (BSVQE) for overall 3D video quality assessment. In the DPQM, we introduce the feature of auto-regressive prediction-based disparity entropy (ARDE) measurement and the feature of energy weighted video content measurement, which are inspired by the free-energy principle and the binocular vision mechanism. In the BSVQE, the binocular summation and difference operations are integrated together with the fusion natural scene statistic measurement and the ARDE measurement to reveal the key influence from texture and disparity. Experimental results on three stereoscopic video databases demonstrate that our method outperforms state-of-theart SVQA algorithms for both symmetrically and asymmetrically distorted stereoscopic video pairs of various distortion types.

I. Introduction

Three-dimensional television (3D-TV) provides an entirely new viewing experience. However, there are still many quality issues in stereoscopic contents. Therefore, stereoscopic image/video quality assessment is an important and challenging research problem, which attracts a lot of attentions [1]. Stereoscopic image/video quality assessment contains multi-dimensional qualities. Three basic perceptual quality dimensions, namely picture quality, depth quality and visual discomfort, are identified in [2] to synthetically affect the overall quality of experience (QoE) of 3D image/video. It is essential to evaluate stereoscopic contents in all of the dimensions, not simply in picture quality. In other words, the ultimate goal of stereoscopic video quality assessment is to develop an evaluation criterion that reflects total user experience. Moreover, ocular and cognitive conflicts may cause visual fatigue and discomfort [3], which include vergence-accommodation conflict [4], cognitive integration of conflicting depth cues, and so on. In addition, visual fatigue and discomfort are also caused by display difference, viewing distance, duration of viewing, and subject variation [5], [6]. Meanwhile, several studies and proposed models on 3D visual discomfort have arisen recently. For example in [7], a study on the relationship of 3D video characteristics, eye blinking rate, and visual discomfort is conducted. In [8], a new concept named the percentage of un-linked pixels map (PUP map) is built to predict the degree of 3D visual discomfort. Basically, the experimental methods and models of visual discomfort are quite independent of that of image quality and depth quality. Consequently, when viewing stereoscopic contents, apart from visual discomfort, image quality and depth quality are two significant aspects of overall 3D QoE which this paper concentrates on.