Cubemap-Based Perception-Driven Blind Quality Assessment for 360-degree Images

image can be represented with different formats, such as the equirectangular projection (ERP) image, viewport images or spherical image, for its different processing procedures and applications. Accordingly, the 360-degree image quality assessment (360-IQA) can be performed on these different formats. However, the performance of 360-IQA with the ERP image is not equivalent with those with the viewport images or spherical image due to the over-sampling and the resulted obvious geometric distortion of ERP image. This imbalance problem brings challenge to ERP image based applications, such as 360-degree image/video compression and assessment. In this paper, we propose a new blind 360-IQA framework to handle this imbalance problem. In the proposed framework, cubemap projection (CMP) with six inter-related faces is used to realize the omnidirectional viewing of 360-degree image. A multi-distortions visual attention quality dataset for 360-degree images is firstly established as the benchmark to analyze the performance of objective 360-IQA methods. Then, the perception-driven blind 360-IQA framework is proposed based on six cubemap faces of CMP for 360-degree image, in which human attention behavior is taken into account to improve the effectiveness of the proposed framework. The cubemap quality feature subset of CMP image is first obtained, and additionally, attention feature matrices and subsets are also calculated to describe the human visual behavior. Experimental results show that the proposed framework achieves superior performances compared with state-of-the-art IQA methods, and the cross dataset validation also verifies the effectiveness of the proposed framework. In addition, the proposed framework can also be combined with new quality feature extraction method to further improve the performance of 360-IQA. All of these demonstrate that the proposed framework is effective in 360-IQA and has a good potential for future applications.

I. Introduction

Omnidirectional image, also known as 360-degree image, can provide users with unique interactive and immersive visual experience through head mounted device (HMD) [1]. To get high quality 360-degree image, multiple source images are usually captured with multiple cameras/lenses and then stitched together to generate a spherical projection (SP) image, which is different from traditional two-dimensional (2D) plane image [2]. 360-degree image has various representations based on its omnibearing, such as the equirectangular projection (ERP) image, cubemap projection (CMP) image, viewport images, SP image, and so on. Thus, how to select appropriate evaluation object for objective 360-degree image quality assessment (360-IQA) has become an important issue [3]. Moreover, considering that the main purposes of 360-IQA are to measure and optimize the 360-degree image/video systems, including capturing, coding, etc., the design of 360-IQA framework will be more meaningful if it can be unified with coding standards.