ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution

Diffusion models have gained significant popularity for image-to-image translation tasks. Previous efforts applying diffusion models to image super-resolution have demonstrated that iteratively refining pure Gaussian noise using a U-Net architecture trained on denoising at various noise levels can yield satisfactory high-resolution images from low-resolution inputs. However, this iterative refinement process comes with the drawback of low inference speed, which strongly limits its applications. To speed up inference and further enhance the performance, our research revisits diffusion models in image super-resolution and proposes a straightforward yet significant diffusion model-based super-resolution method called ACDMSR (accelerated conditional diffusion model for image super-resolution). Specifically, we adopt existing image super-resolution methods and finetune them to provide conditional images from given low-resolution images, which can help to achieve better high-resolution results than just taking low-resolution images as conditional images. Then we adapt the diffusion model to perform super-resolution through a deterministic iterative denoising process, which helps to strongly decline the inference time. We demonstrate that our method surpasses previous attempts in qualitative and quantitative results through extensive experiments conducted on benchmark datasets such as Set5, Set14, Urban100, BSD100, and Manga109. Moreover, our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.

I. Introduction

Single image super-resolution (SISR) has drawn active attention due to its wide applications in computer vision, such as object recognition [1], remote sensing [2], [3] and so on [4], [5], [6], [7], [8], [9], [10]. SISR aims to obtain a high-resolution (HR) image containing great details and textures from a low-resolution (LR) image by a super-resolution method, which is a classic ill-posed inverse problem [11], [12], [13], [14], [15], [16], [17]. To establish the mapping between HR and LR images, lots of CNN-based methods have emerged [18], [19], [20], [21], [22]. These methods focus on designing novel architectures by adopting different network modules, such as residual blocks [23], attention blocks [24], non-local blocks [25], transformer layers [7], [26], and contrastive learning [27], [28]. For optimizing the training process, they prefer to use the MAE or MSE loss (e.g., or ) to optimize the architectures, which often leads to over-smooth results because the above losses provide a straightforward learning objective and optimize for the popular PSNR (peak signal-to-noise-ratio) metric [29], [30], [31], [32], [33].