Speech-Driven Facial Animation with Transfer Learning

In order to move a CG model, it is necessary to manually move it with the conventional method, and the task is extremely difficult and time-consuming. Therefore, there is a method to generate CG animation from the feature points of the tracked face. However, face tracking technology has problems such as the inability to track if the face is hidden, and the need for large-scale equipment to track with high accuracy. In order to solve such problems, research has been conducted to generate facial expressions by using machine learning from speech linked to mouth movement. However, these studies do not take into account the individuality of the user during the learning process. In this paper, we propose a method for performing transfer learning on a trained model for the purpose of generating animations that take into account the user's individuality. Verification experiments based on the proposed method are conducted with data generated with and without transfer learning as an evaluation.