360-Degree Intra Coding Mode for Equirectangular Projection Format Videos

Recent advances in display, networking, and computing technologies have resulted in changing industry focus towards 360-degree/omnidirectional images as witnessed by increased interest in virtual reality (VR) and augmented reality (AR). Numerous curves are generated for 360-degree images due to the lens curvature and projection format. However, in High Efficiency Video Coding (HEVC), the conventional angular intra modes cannot handle them well since only straight lines can be predicted. More enhanced 360-degree image coding is essential for higher efficiency of storage and transmission. Therefore, we propose a new coding mode, called 360-degree intra mode, for predicting the coding units with curves. Experimental results show that our 360-degree intra mode improves the coding efficiency of HEVC by 0.32% on average and up to 0.74% Bj⊘ntegaard delta bit rate reduction.

I. Introduction

With the facilitation of sensors, computing devices, and 5G networks, it is feasible for the provision of immersive media such as Virtual Reality (VR) and Augmented Reality (AR), which primarily demands 360-degree images. Recently, Motion Picture Experts Group (MPEG) has launched the MPEG-I project for the coding representation of immersive media (ISO/IEC 23090) [1], [2]. It is divided into three phases to support three degrees of freedom (3DoF), 3DoF+, and 6DoF experiences for viewing the immersive media, and they are named as Phase 1a, Phase 1b, and Phase 2, respectively [3]-[5]. In particular, the goal of Phase 1a is to provide users with 3DoF experience of yaw, pitch, and roll for watching 360-degree video content [6]. And MPEG-I has defined the Omnidirectional MediA Format (OMAF) which is the first VR system standard to support 3DoF [7]. As shown in Fig. 1, a 360-degree video, captured by a 360-degree video capture device or generated by multiple stitched images, is projected as a rectangular format followed by encoding. This 360-degree video provides 360×180 degrees of field of view. After transmission and decoding, the video is rendered on the sphere, and a user can view the sphere to experience the 360-degree environment through VR devices such as Head-Mounted Displays (HMDs). Thus, it is essential to improve the coding efficiency of 360-degree images. One of the popular projection formats is equirectangular format (ERP). This format provides one single rectangular image of 360 degrees horizontally and 180 degrees vertically for the ease of encoding. But it also distorts the content and generates many curve based structures within the image.