Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution

Real-world face super-resolution (SR) is a highly ill-posed image restoration task. The fully-cycled Cycle-GAN architecture is widely employed to achieve promising performance on face SR, but is prone to produce artifacts upon challenging cases in real-world scenarios, since joint participation in the same degradation branch will impact final performance due to huge domain gap between real-world and synthetic LR ones obtained by generators. To better exploit the powerful generative capability of GAN for real-world face SR, in this paper, we establish two independent degradation branches in the forward and backward cycle-consistent reconstruction processes, respectively, while the two processes share the same restoration branch. Our Semi-Cycled Generative Adversarial Networks (SCGAN) is able to alleviate the adverse effects of the domain gap between the real-world LR face images and the synthetic LR ones, and to achieve accurate and robust face SR performance by the shared restoration branch regularized by both the forward and backward cycle-consistent learning processes. Experiments on two synthetic and two real-world datasets demonstrate that, our SCGAN outperforms the state-of-the-art methods on recovering the face structures/details and quantitative metrics for real-world face SR. The code will be publicly released at https://github.com/HaoHou-98/SCGAN.

I. Introduction

Face is of central importance for human identity recognition. The low-resolution (LR) face images captured by camera sensors would largely degrade the corresponding identity information. Face super-resolution (SR) aims to estimate high-resolution (HR) face images from LR ones, to improve the image quality and performance of subsequent identity recognition tasks [1], [2], [3]. This task is very challenging upon complex real-world scenarios, where the degradation kernel is usually unknown. Traditional face SR methods can be roughly divided into local patch-based methods [4], [5], [6], global image-based methods [7], [8], [9], and hybrid methods taking advantage of global image consistency and local patch sparsity [10], [11], [12], [13]. However, these hand-crafted methods could hardly achieve satisfactory results upon diverse real-world scenarios [14].