Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey

This paper introduces a novel benchmark for efficient image upscaling as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to up scale compressed images from 540p to native 4K resolution (4x factor) in real-time on commercial GPUs. For this, we use a diverse test set containing diverse 4K images ranging from digital art to gaming and photography. The images are compressed using the modern AVIF codec, instead of JPEG. All the proposed methods improve PSNR fidelity over Lanczos interpolation, and process images under 30ms. Out of the 160 participants, 25 teams submitted their code. This survey considers only the most novel solutions models, making it the most comprehensive benchmark on real-time SR of compressed images using modern codecs.

1. Introduction

Single image super-resolution (SR) methods generate a high-resolution (HR) image from a single degraded low-resolution (LR) image. This ill-posed problem was initially solved using interpolation methods. However, SR is now commonly approached through the use of deep learning [6], [30], [45]. Image SR assumes that the LR image is obtained through a degradation processes. This can be expressed as:

$$\begin{equation*}{\mathbf{y}} = ({\mathbf{x}}*{\mathbf{k}}){ \downarrow _s},\tag{1}\end{equation*}$$ where * represents the convolution operation between the LR image and the blur kernel, and ↓_s is the down-sampling operation with respective down-sampling factor ×s (e.g. ×2, ×3, ×4, ×8).