Attribute-Guided Sketch Generation

Facial attributes are important since they provide a detailed description and determine the visual appearance of human faces. In this paper, we aim at converting a face image to a sketch while simultaneously generating facial attributes. To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation. The two generators form a W-shaped network (W-net) and they are trained jointly with a weight-sharing constraint. Additionally, we also propose two novel discriminators, the residual one focusing on attribute generation and the triplex one helping to generate realistic looking sketches. To validate our model, we have created a new large dataset with 8,804 images, named the Attribute Face Photo & Sketch (AFPS) dataset which is the first dataset containing attributes associated to face sketch images. The experimental results demonstrate that the proposed network (i) generates more photo-realistic faces with sharper facial attributes than baselines and (ii) has good generalization capability on different generative tasks.

I. Introduction

Recently, there has been a new trend in computer vision to use machines to express the "creativity" of art. Novel and never-seen-before images can be generated by inverting the convolution process in CNN ("upconvolution" or "deconvolution"), which gives such networks the ability to "dream" [23] and to generate images. To implement these tasks, deep generative models have been usually adopted, and these networks have also been employed for face-to-sketch translation. These deep models can be roughly divided into two categories, namely, the Generative Adversarial Networks (GANs) [4] and Variational AutoEncoders (VAEs) [11], [14], [18]. In GANs, there are two subnetworks, a generator and a discriminator, which play the two-player minimax game with a value function [4]. The generator acts as a mapping function to convert an input image to the generated image so that it fools the discriminator, which is trained to distinguish the generated images from the input real images. In this work, we rely on GANs as the basis to implement the face-to-sketch translation model.