Face Frontalization Method with 3D Technology and a Side-Face-Alignment Generative Adversarial Networks

When the pose of a face changes, the facial features may be misaligned or shifted, thus reducing the quality of facial details. Therefore, face images with different angles affect face recognition, expression synthesis, style migration, and other related studies. To address this problem, we propose a face frontalization conversion model based on 3D technology and generative adversarial networks. The model first performs face fitting via 3DFFA, which is rotated and rendered in 3D space to generate training data pairs. These paired training data are then fed into a generative adversarial network. We use a modified CirGAN generator and a multiscale discriminator for training, and non-frontal face images from natural environments are fed into the generator to generate frontal face images. The multiscale discriminator is used to judge the quality of the generated images. Next, the Facenet and MTCNN are used to extract the features of the converted face and obtain the recognition results of the different angles of the face to verify the model's conversion effect. We experimented with the model on Multiple and CFP datasets, and the results show that the non-frontal face recognition results are improved compared with VGG-FACE, HPN, TP-GAN, and CAPGGAN.

I. Introduction

In computer vision, face frontalization technology plays an increasingly important role in today's society [1], such as facial recognition, expression synthesis, style migration, etc., and it has always been a hotspot of research. Although its technology has made remarkable progress, effectively dealing with the variation of large gestures of faces is still a significant challenge within the industry [2]. For example, traditional face recognition methods are usually performed in experimental environments that rely on frontal face images. In real-world environments, however, faces often present multiple angles due to different head postures, and the texture information for their facial recognition is not as recognizable as frontal facial texture information [3], along with problems such as facial deformation of the face and displacement of the five views. This directly affects the accuracy and robustness of the recognition system.