Facial Expression Editing in Face Sketch Using Shape Space Theory

Facial expression editing in face sketch is an important and challenging problem in computer vision community as facial animation and modeling. For criminal investigation and portrait drawing, automatic expression editing tools for face sketch improve work efficiency obviously and reduce professional requirements for users. In this paper, we propose a novel method for facial expression editing in face sketch using shape space theory. The new facial expressions in the sketch images can be regenerated automatically. The method includes two components: 1) face sketch modeling; 2) expression editing. The face sketch modeling constructs 3D face sketch data from 3D facial database to match the 2D face sketch. Using facial landmarks, the "shape" of the face sketch is represented in shape space. The shape space is a manifold space which removes the rigid transform group. In shape space, the accurate 3D face sketch model is obtained which is consistent to the original 2D face sketch. For expression editing, we change the parameters of 3D face sketch model in the shape space to obtain new expressions. The expression transfer in 3D face sketch model can be mapped into the 2D face sketch. The advantages of our method are: full-automatic in modeling process; no requirements of drawing skills to user and friendly interaction; robustness to head poses and different scales. In experiments, we use the 3D facial database, FaceWareHouse, to construct the 3D face sketch model and use face sketch images from database: CUHK Face sketch Database (CUFS) to show the performance of expression editing. Experimental results demonstrate that our method can effectively edit facial expressions in face sketch with high consistency and fidelity.

I. Introduction

As a type of facial data, a face sketch has irreplaceable value in many applications such as law enforcement assist and portrait drawing. The face sketch images are obtained from professional painter or face photo based synthesis method in most cases. Once the sketch drawing process is complete, it will be difficult to edit the face sketch, especially to expression modification. The expression editing in face sketch by artist require repetitive and complex workload. For some users without painting skills, it is impossible to change expression in face sketch artificially. Therefore, the automatic expression editing tools for face sketch are required in practical. The tools improve the work efficiency of artist drawing and provide ability for nonprofessional users to edit the expression in face sketch to obtain the ideal face sketch image.