Ensemble Methods for Robust 3D Face Recognition Using Commodity Depth Sensors

In this paper we introduce a new dataset and pose invariant sampling method and describe the ensemble methods used for recognizing faces in 3D scenes, captured using commodity depth sensors. We use the 3D SIFT key point detector to take advantage of the similarities between faces, which leads to a set of points of interest based on the curvature of the face. For all key points, features are extracted using a 3D feature descriptor. Then, a variable-sized amount of features are generated per each 3D face image. The first ensemble method we constructed uses a K-nearest neighbors classifier to classify each key point-sampled feature vector as belonging to one of the subjects recorded in our dataset. All votes over all key points are combined. In the second ensemble technique, the key points are clustered with K-means, using the feature vectors and approximated sampling positions relative to the face. This leads to a set of experts that specialize for a specific region. Then a K-nearest neighbors classifier is trained on the examples falling in each expert's specialized region. Finally, for a new 3D face image, votes from all experts are combined in a sum ensemble technique to categorize the 3D face. We also introduce 6 new "real world" datasets with different variances: 3 types of 3D rotations, distance to sensor, expressions, and an all-in-one dataset. The results show very high cross validation accuracies for the same type of variance. In addition, 36 variance specific pair-tests in which the system is trained on one dataset and tested on a completely different dataset also show encouraging results.

I. Introduction

Face perception is perhaps the most highly developed visual skill in humans. The evolution of our perceptual systems has taken a very long time to reach current capabilities. Machines do not necessarily have to possess the same type of sensors as we do, hence we make use of commodity infrared depth sensors, which do not require intensive preprocessing for extracting shape cues. Furthermore, depth sensors such as the Kinect offer several advantages compared to normal cameras, such as robustness to different lighting conditions and extreme pose angles.