NTIRE 2023 Challenge on 360° Omnidirectional Image and Video Super-Resolution: Datasets, Methods and Results

This report introduces two high-quality datasets Flickr360 and ODV360 for omnidirectional image and video super-resolution, respectively, and reports the NTIRE 2023 challenge on 360° omnidirectional image and video super-resolution. Unlike ordinary 2D images/videos with a narrow field of view, omnidirectional images/videos can represent the whole scene from all directions in one shot. There exists a large gap between omnidirectional image/video and ordinary 2D image/video in both the degradation and restoration processes. The challenge is held to facilitate the development of omnidirectional image/video super-resolution by considering their special characteristics. In this challenge, two tracks are provided: one is the omnidirectional image super-resolution and the other is the omnidirectional video super-resolution. The task of the challenge is to super-resolve an input omnidirectional image/video with a magnification factor of ×4. Realistic omnidirectional downsampling is applied to construct the datasets. Some general degradation(e.g., video compression) is also considered for the video track. The challenge has 100 and 56 registered participants for those two tracks. In the final testing stage, 7 and 3 participating teams submitted their results, source codes, and fact sheets. Almost all teams achieved better performance than baseline models by integrating omnidirectional characteristics, reaching compelling performance on our newly collected Flickr360 and ODV360 datasets.

1. Introduction

The 360° or omnidirectional images/videos can provide users with an immersive and interactive experience, and have received much research attention with the popularity of AR/VR applications. Unlike planar 2D images/videos with a narrow field of view (FoV), 360° images/videos can represent the whole scene in all directions. However, 360° images/videos suffer from the lower angular resolution problem since they are captured by the fisheye lens with the same sensor size for capturing planar images. Although the 360° images/videos are high-resolution, their details are usually missing. In many application scenarios, increasing the resolution of 360° images/videos is highly demanded to achieve higher perceptual quality and boost the performance of downstream tasks.