ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations

Precise representations of 3D faces are beneficial to various computer vision and graphics applications. Due to the data discretization and model linearity, however, it remains challenging to capture accurate identity and expression clues in current studies. This paper presents a novel 3D morphable face model, namely ImFace, to learn a nonlinear and continuous space with implicit neural representations. It builds two explicitly disentangled deformation fields to model complex shapes associated with identities and expressions, respectively, and designs an improved learning strategy to extend embeddings of expressions to allow more diverse changes. We further introduce a Neural Blend-Field to learn sophisticated details by adaptively blending a series of local fields. In addition to ImFace, an effective pre-processing pipeline is proposed to address the issue of watertight input requirement in implicit representations, enabling them to work with common facial surfaces for the first time. Extensive experiments are performed to demonstrate the superiority of ImFace.

1. Introduction

3D Morphable Face Models (3DMMs) are well-reputed statistical models, established by learning techniques upon prior distributions of facial shapes and textures from a set of samples with dense correspondence, aiming at rendering realistic faces of a high variety. Since a morphable representation is unique across different downstream tasks where the geometry and appearance are separately control-lable, 3DMMs are pervasively exploited in many face analysis applications in the field of computer vision, computer graphics, biometrics, and medical imaging [2], [8], [27],[50].