Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer

Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) by accessing both the source and target data. Considering privacy policies or transmission restrictions of source data in practical scenarios, we propose a SOurce-free Domain Adaptation framework for image SR (SODA-SR) to address this issue, i.e., adapt a source-trained model to a target domain with only unlabeled target data. SODA-SR leverages the source-trained model to generate refined pseudo-labels for teacher-student learning. To better utilize pseudo-labels, we propose a novel wavelet-based augmentation method, named Wavelet Augmentation Transformer (WAT), which can be flexibly incorporated with existing networks, to implicitly produce useful augmented data. WAT learns low-frequency information of varying levels across diverse samples, which is aggregated efficiently via deformable attention. Furthermore, an uncertainty-aware self-training mechanism is proposed to improve the accuracy of pseudo-labels, with inaccurate predictions being rectified by uncertainty estimation. To acquire better SR results and avoid overfitting pseudo-labels, several regularization losses are proposed to constrain target LR and SR images in the frequency domain. Experiments show that without accessing source data, SODA-SR outperforms state-of-the-art UDA methods in both synthetic→real and real→real adaptation settings, and is not constrained by specific network architectures.

1. Introduction

Single image super-resolution (SISR), which is a fundamental task in low-level vision, aims to reconstruct a high-resolution (HR) image from its low-resolution (LR) counterpart. In recent years, owing to the thriving advancements in deep learning, numerous deep learning-based approaches have been applied to SISR, culminating in significant break-throughs in this task. Predominantly, these methods employ Convolutional Neural Networks (CNNs) [8], [31] or Vision Transformers (ViTs) [9], [37] as their architectural foundation. However, the majority are trained on synthetic datasets that generate LR images using simplistic and predetermined degradation kernels (e.g., bicubic). Figure 1.

PSNR vs. the usage of source data on the DRealSR [56] dataset. The less source data a method uses, the more restrictions it faces. SFDA and SSL represent source-free domain adaption and self-supervised learning methods respectively.