Exploring Robust Pose Invariant Face Recognition with Vision Transformers: A Multi-Modal Study

This paper investigates the application of vision transformers (ViT) to face recognition tasks, focusing on their robustness against pose variations in multi-modal settings. Leveraging the advantages of ViT, such as global context modeling and attention mechanisms, we evaluate their performance using RGB and depth face modalities. To facilitate this investigation, we propose a new multi-view and multi-modal dataset, featuring various pose variations, ranging from slight to extreme, with incorporated facial expressions. Four robust pose invariant evaluation protocols are established for comprehensive analysis. Three variants of ViT models (Base, Small, and Tiny) are trained and evaluated. Our findings indicate that ViT-Base achieves 97.11 % accuracy for RGB faces, while ViT-Small outperforms all RGB Face Transformers in depth modality, achieving 99.02% accuracy across all pose protocols. Importantly, our empirical study highlights the enhanced robustness of 3D depth maps for pose invariant face recognition tasks, irrespective of the deep learning techniques employed. These insights contribute to the understanding of ViT models in face recognition and open avenues for future research in this domain.