Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report

This paper introduces a novel benchmark for efficient up-scaling as part of the NTIRE 2023 Real-Time Image Super-Resolution (RTSR) Challenge, which aimed to upscale images from 720p and 1080p resolution to native 4K (×2 and ×3 factors) in real-time on commercial GPUs. For this, we use a new test set containing diverse 4K images ranging from digital art to gaming and photography. We assessed the methods devised for 4K SR by measuring their runtime, parameters, and FLOPs, while ensuring a minimum PSNR fidelity over Bicubic interpolation. Out of the 170 participants, 25 teams contributed to this report, making it the most comprehensive benchmark to date and showcasing the latest advancements in real-time SR.

1. Introduction

Single image super-resolution (SR) refers to the process of generating a high-resolution (HR) image from a single degraded low-resolution (LR) image. This ill-posed problem was initially solved using interpolation methods [28], [77]–[79]. However, with the emergence of deep learning, SR is now commonly approached through the use of deep neural networks [17], [24], [49], [56], [57], [84], [88], [99]. Image SR assumes that the LR image is obtained through two major degradation processes: blurring and down-sampling. This can be expressed as: $$\begin{equation*}{\mathbf{y}} = ({\mathbf{x}} * {\mathbf{k}}) \downarrow s,\tag{1}\end{equation*}$$ where ∗ represents the convolution operation between the LR image and the blur kernel, and ↓_s is the down-sampling operation with respective down-sampling factor ×s. Most SR methods are built around the Bicubic model [77], [78] with various down-scaling factors (e.g. ×2, ×3, ×4, ×8).