Real-World Image Super-Resolution by Exclusionary Dual-Learning | IEEE Journals & Magazine | IEEE Xplore

Real-World Image Super-Resolution by Exclusionary Dual-Learning


Abstract:

Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input, has recently received considera...Show More

Abstract:

Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input, has recently received considerable attention with regard to its tremendous application potentials. Although deep learning-based methods have achieved promising restoration quality on real-world image super-resolution datasets, they ignore the relationship between L1- and perceptual- minimization and roughly adopt auxiliary large-scale datasets for pre-training. In this paper, we discuss the image types within a corrupted image and the property of perceptual- and Euclidean- based evaluation protocols. Then we propose a method, Real-World image Super-Resolution by Exclusionary Dual-Learning (RWSR-EDL) to address the feature diversity in perceptual- and L1- based cooperative learning. Moreover, a noise-guidance data collection strategy is developed to address the training time consumption in multiple datasets optimization. When an auxiliary dataset is incorporated, RWSR-EDL achieves promising results and repulses any training time increment by adopting the noise-guidance data collection strategy. Extensive experiments show that RWSR-EDL achieves competitive performance over state-of-the-art methods on four in-the-wild image super-resolution datasets.
Published in: IEEE Transactions on Multimedia ( Volume: 25)
Page(s): 4752 - 4763
Date of Publication: 10 June 2022

ISSN Information:

Funding Agency:

References is not available for this document.

I. Introduction

Real-world image super-resolution (RealSR) aims at restoring in-the-wild images collected from poor-quality sensors with unknown degraded kernels. Since RealSR realizes image restoration under real-world scenarios, it plays a remarkable role in many human-centric applications, such as mobile photo enhancement, automatic pilot, etc.

Select All
1.
Y. Yan et al., "Fine-grained attention and feature-sharing generative adversarial networks for single image super-resolution", IEEE Trans. Multimedia, vol. 24, pp. 1473-1487, 2022.
2.
K. Jiang et al., "Dual-path deep fusion network for face image hallucination", IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 1, pp. 378-391, Jan. 2022.
3.
C. Tian et al., "Coarse-to-fine CNN for image super-resolution", IEEE Trans. Multimedia, vol. 23, pp. 1489-1502, 2021.
4.
M. Li, Z. Zhang, J. Yu and C. W. Chen, "Learning face image super-resolution through facial semantic attribute transformation and self-attentive structure enhancement", IEEE Trans. Multimedia, vol. 23, pp. 468-483, 2021.
5.
Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity", IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
6.
M. Lucic, K. Kurach, M. Michalski, S. Gelly and O. Bousquet, "Are GANs created equal? A large-scale study", Proc. Adv. Neural Inf. Process. Syst., pp. 700-709, 2018.
7.
R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 586-595, 2018.
8.
K.-Y. Lin and G. Wang, "Hallucinated-IQA: No-reference image quality assessment via adversarial learning", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 732-741, 2018.
9.
A. Lugmayr et al., "Aim 2019 challenge on real-world image super-resolution: Methods and results", Proc. IEEE Int. Conf. Comput. Vis. Workshops, pp. 3575-3583, 2019.
10.
A. Lugmayr et al., "NTIRE 2020 challenge on real-world image super-resolution: Methods and results", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, pp. 2058-2076, 2020.
11.
C. Chen, Z. Xiong, X. Tian, Z. Zha and F. Wu, "Camera lens super-resolution", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1652-1660, 2019.
12.
J. Cai, H. Zeng, H. Yong, Z. Cao and L. Zhang, "Toward real-world single image super-resolution: A new benchmark and a new model", Proc. IEEE Int. Conf. Comput. Vis., pp. 3086-3095, 2019.
13.
P. Wei et al., "Component divide-and-conquer for real-world image super-resolution", Proc. Eur. Conf. Comput. Vis., pp. 101-117, 2020.
14.
B. Lim, S. Son, H. Kim, S. Nah and K. M. Lee, "Enhanced deep residual networks for single image super-resolution", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, pp. 1132-1140, 2017.
15.
Y. Zhang et al., "Image super-resolution using very deep residual channel attention networks", Proc. Eur. Conf. Comput. Vis., pp. 286-301, 2018.
16.
Y. Zhang, Y. Tian, Y. Kong, B. Zhong and Y. Fu, "Residual dense network for image restoration", IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 7, pp. 2480-2495, Jul. 2020.
17.
R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang and L. Zhang, "NTIRE 2017 challenge on single image super-resolution: Methods and results", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, pp. 114-125, 2017.
18.
Y. Wei et al., "Unsupervised real-world image super resolution via domain-distance aware training", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 13385-13394, 2021.
19.
C. Dong, C. C. Loy, K. He and X. Tang, "Learning a deep convolutional network for image super-resolution", Proc. Eur. Conf. Comput. Vis., pp. 184-199, 2014.
20.
K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image recognition", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 770-778, 2016.
21.
Z. Luo et al., "EBSR: Feature enhanced burst super-resolution with deformable alignment", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 471-478, 2021.
22.
Y. Zhang et al., "Image super-resolution using very deep residual channel attention networks" in Proc. Eur. Conf. Comput. Vis., pp. 286-301, 2018.
23.
X. Wang et al., "ESRGAN: Enhanced super-resolution generative adversarial networks", Proc. Eur. Conf. Comput. Vis. Workshops, pp. 63-79, 2018.
24.
X. Hu et al., "Meta-USR: A unified super-resolution network for multiple degradation parameters", IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 9, pp. 4151-4165, Sep. 2021.
25.
Y. Mei et al., "Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 5689-5698, 2020.
26.
K. Jiang, Z. Wang, P. Yi and J. Jiang, "Hierarchical dense recursive network for image super-resolution", Pattern Recognit., vol. 107, 2020.
27.
A. Shocher, N. Cohen and M. Irani, "Zero-shot super-resolution using deep internal learning", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3118-3126, 2018.
28.
S. Bell-Kligler, A. Shocher and M. Irani, "Blind super-resolution kernel estimation using an internal-GAN", Proc. Adv. Neural Inf. Process. Syst., pp. 3086-3095, 2019.
29.
M. Fritsche, S. Gu and R. Timofte, "Frequency separation for real-world super-resolution", Proc. IEEE Int. Conf. Comput. Vis. Workshops, pp. 3599-3608, 2019.
30.
Y. Pang et al., "FAN: Frequency aggregation network for real image super-resolution", Proc. Eur. Conf. Comput. Vis., pp. 468-483, 2020.
Contact IEEE to Subscribe

References

References is not available for this document.