Pose Aware RGBD-based Face Recognition System With Hierarchical Bilinear Pooling

The accuracy of face recognition is not satisfactory in large poses. It is because the existing face recognition methods rely only on RGB images, which lose information in the large head pose. Therefore, we utilize depth images containing 3D dimension information to complement the loss. In this work, we propose a pose-invariant RGBD face recognition method with the Hierarchical Bilinear Pooling module (HBP) and Pose-Aware Module (PAM). Most of the existing methods consider the final layer of the model. However, the final layer does not contain enough facial information, such as the eyes, nose, and mouth. Thus, we propose the HBP module to fuse high-level layers which can focus on important facial areas but with different perspectives than the final layer. To reduce the effect of head poses, we propose the PAM module via mapping the profile faces to frontal faces in feature space. We verify our method on the datasets proposed by [1] and KinectFaceDB [2]. Our method achieves 1.3% improvement and has 6.1 million fewer parameters than the method in [3].