Efficient Panoramic Video Coding for Immersive Metaverse Experience

Efficient panoramic video coding plays a crucial role in the metaverse and Web 3.0 by enhancing content delivery, accessibility, and scalability. However, panoramic video is viewed in the spherical domain, while it is coded in the typical two-dimensional plane. Such a framework renders the compression-distortion metric unable to align well with the spherical-distortion perceived by viewers, resulting in inefficient rate-distortion optimization (RDO) in the coding process. Additionally, independent RDO on an individual panoramic video frame is also inefficient as it disregards the distortion propagation caused by the inter-prediction of video coding. To address these issues, a temporal-dependent spherical-distortion model is proposed for efficient panoramic video coding. Using the geometric projection principle, an independent mapping model between the spherical-distortion and the compression-distortion is first established for individual frames. Subsequently, the temporal-dependent spherical-distortion model for consecutive frames is deduced based on the inter-prediction structure. This model is then employed to guide the RDO process for panoramic video coding. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art methods, achieving an average bitrate reduction of 4.2% compared to the reference software VTM with the 360Lib extension.