Template Inversion Attack against Face Recognition Systems using 3D Face Reconstruction

Face recognition systems are increasingly being used in different applications. In such systems, some features (also known as embeddings or templates) are extracted from each face image. Then, the extracted templates are stored in the system’s database during the enrollment stage and are later used for recognition. In this paper, we focus on template inversion attacks against face recognition systems and introduce a novel method (dubbed GaFaR) to reconstruct 3D face from facial templates. To this end, we use a geometry-aware generator network based on generative neural radiance fields (GNeRF), and learn a mapping from facial templates to the intermediate latent space of the generator network. We train our network with a semi-supervised learning approach using real and synthetic images simul taneously. For the real training data, we use a Generative Adversarial Network (GAN) based framework to learn the distribution of the latent space. For the synthetic training data, where we have the true latent code, we directly train in the latent space of the generator network. In addition, during the inference stage, we also propose optimization on the camera parameters to generate face images to improve the success attack rate (up to 17.14% in our experiments). We evaluate the performance of our method in the whitebox and blackbox attacks against state-of-the-art face recognition models on the LFW and MOBIO datasets. To our knowledge, this paper is the first work on 3D face reconstruction from facial templates. The project page is available at: https://www.idiap.ch/paper/gafar

1. Introduction

Automatic face recognition (FR) has become a well-known biometric authentication tool which has been widely used in different applications, including smartphone locks, border controls, etc. In such systems, generally, some features (also referred to as facial embeddings or facial templates) are extracted from users and stored in the system’s database during the enrollment stage. In the recognition stage, either verification or identification, similar features are extracted from the user and are compared with the features in the system’s database. Therefore, facial features play the main role in automatic face recognition systems and convey important information about users.