Towards Real-World Burst Image Super-Resolution: Benchmark and Method

Despite substantial advances, single-image super-resolution (SISR) is always in a dilemma to reconstruct high-quality images with limited information from one input image, especially in realistic scenarios. In this paper, we establish a large-scale real-world burst super-resolution dataset, i.e., RealBSR, to explore the faithful reconstruction of image details from multiple frames. Furthermore, we introduce a Federated Burst Affinity network (FBAnet) to investigate non-trivial pixel-wise displacements among images under real-world image degradation. Specifically, rather than using pixel-wise alignment, our FBAnet employs a simple homography alignment from a structural geometry aspect and a Federated Affinity Fusion (FAF) strategy to aggregate the complementary information among frames. Those fused informative representations are fed to a Transformer-based module of burst representation decoding. Besides, we have conducted extensive experiments on two versions of our datasets, i.e., RealBSR-RAW and RealBSR-RGB. Experimental results demonstrate that our FBAnet outperforms existing state-of-the-art burst SR methods and also achieves visually-pleasant SR image predictions with model details. Our dataset, codes, and models are publicly available at https://github.com/yjsunnn/FBANet.

1. Introduction

As a fundamental research topic, Super-Resolution (SR) attracts long-standing substantial interest, which targets high-resolution (HR) image reconstruction from a single or a sequence of low-resolution (LR) observations. In recent years, we have witnessed the prosperity of Single Image Super-Resolution (SISR), e.g., SRCNN [7], EDSR [19], SRGAN [16], RDN [33] and ESRGAN [28]. Nevertheless, SISR intrinsically suffers from a limited capacity of restoring details from only one LR image, typically yielding oversmooth LR predictions, especially for large-scale factors. With real detailed sub-pixel displacement information, Multi-Frame Super-Resolution (MFSR) [31], [1], [2], [21], [20] provides a promising potential to reconstruct the high-quality image from multiple LR counterparts, which is valuable for many sensitive realistic applications, e.g., medical imaging, and remote satellite sensing.