FDSR: An Interpretable Frequency Division Stepwise Process Based Single-Image Super-Resolution Network

Deep learning has excelled in single-image super-resolution (SISR) applications, yet the lack of interpretability in most deep learning-based SR networks hinders their applicability, especially in fields like medical imaging that require transparent computation. To address these problems, we present an interpretable frequency division SR network that operates in the image frequency domain. It comprises a frequency division module and a step-wise reconstruction method, which divides the image into different frequencies and performs reconstruction accordingly. We develop a frequency division loss function to ensure that each reconstruction module (ReM) operates solely at one image frequency. These methods establish an interpretable framework for SR networks, visualizing the image reconstruction process and reducing the black box nature of SR networks. Additionally, we revisited the subpixel layer upsampling process by deriving its inverse process and designing a displacement generation module. This interpretable upsampling process incorporates subpixel information and is similar to pre-upsampling frameworks. Furthermore, we develop a new ReM based on interpretable Hessian attention to enhance network performance. Extensive experiments demonstrate that our network, without the frequency division loss, outperforms state-of-the-art methods qualitatively and quantitatively. The inclusion of the frequency division loss enhances the network’s interpretability and robustness, and only slightly decreases the PSNR and SSIM metrics by an average of 0.48 dB and 0.0049, respectively.

I. Introduction

Single-image super-resolution reconstruction (SISR) is a vital branch of computer vision (CV) [1], [2] that focuses on reconstructing low-resolution (LR) images into high-resolution (HR) images [3]. SISR is applicable to various real-world scenarios, including medical image processing [4] and restoring old photos [5]. Moreover, it serves to enhance other CV tasks through preprocessing [3], [6]. Traditional SR reconstruction methods encompass interpolation-based techniques like bicubic interpolation [7], reconstruction-based approaches such as reverse iterative projection [8], and learning-based methods like sparse representation [9].