Frontal to profile face verification in the wild

We have collected a new face data set that will facilitate research in the problem of frontal to profile face verification `in the wild'. The aim of this data set is to isolate the factor of pose variation in terms of extreme poses like profile, where many features are occluded, along with other `in the wild' variations. We call this data set the Celebrities in Frontal-Profile (CFP) data set. We find that human performance on Frontal-Profile verification in this data set is only slightly worse (94.57% accuracy) than that on Frontal-Frontal verification (96.24% accuracy). However we evaluated many state-of-the-art algorithms, including Fisher Vector, Sub-SML and a Deep learning algorithm. We observe that all of them degrade more than 10% from Frontal-Frontal to Frontal-Profile verification. The Deep learning implementation, which performs comparable to humans on Frontal-Frontal, performs significantly worse (84.91% accuracy) on Frontal-Profile. This suggests that there is a gap between human performance and automatic face recognition methods for large pose variation in unconstrained images.

1. Introduction

Face recognition for unconstrained images is a challenging problem, due to variation in pose, illumination, expression, age and occlusion. A significant challenge of pose variation occurs when features from the whole face are not visible. These situations appear often in many real world scenarios like, surveillance and photo-tagging, where it is quite natural for a person not to face the camera. In this paper we plan to study the effect of pose variation, isolated as a factor, in the presence of other ‘in the wild’ variations. One such case of interest is matching a frontal face facing a camera to a profile face facing away from the camera. The features available in both these views vary significantly and are therefore difficult to match.