Speech-Driven 3D Facial Animation with Mesh Convolution

Facial animation has always been a high-profile problem in computer graphics and computer vision and has been extensively studied. But speech-driven realistic 3D face animation remains a challenging problem because of the complex geometry of 3D face models and the small size of 3D audiovisual datasets. Previous studies have focused on using fully connected layers in the last layer of the network to regress latent representations of speech to 3D face vertex offset points without considering the geometry of the 3D model. To address this limitation, we introduce mesh convolution in the decoder to better learn the high-level mapping relationship between speech latent features and 3D face models, enabling natural and realistic 3D face animations. Finally, our experimental results on public datasets verify the advantages and disadvantages of the proposed network compared with existing models.