Real-Time Masked Face Recognition in the Wild with few shots

Facial masks have become essential in our daily lives, especially at airports, hospitals, and other public places. Post-pandemic, we observe people wearing masks to protect themselves from dust, viral infections, and pollution. While face recognition is a well-studied problem, Masked Face Recognition (MFR) is challenging due to the prominent part of the face being missing or occluded by the mask. In the literature, fine-tuned deep-learning models for MFR have been proposed. Many individual deep-learning networks have been built and fine-tuned to work with MFR datasets. However, they struggle to identify real-time masked faces in uncontrolled environments. This paper introduces an approach for real-time MFR detection based on Few-Shot Learning (FSL) using the Labeled Faces in the Wild (LFW) dataset. Our model excels at identifying both unmasked and masked faces in real-time scenarios. The current practices in normal face recognition or masked face recognition datasets make it challenging to register new masked faces and identify them in real-time. To address this problem, we propose a customized FSL dataset that combines LFW and MLFW (Masked LFW) and includes an augmented dataset featuring both face-to-mask and mask-to-face transformations. In real-time face registration and identification scenarios, it is crucial to train our model using only a few shots. The few-shot data used for registering faces are augmented using a pre-trained face completion model. This augmented data is then processed by existing deep-face models to generate embeddings, which are further trained using an SVM classifier for masked face identification. In comparison to baseline models, our proposed approach for masked face identification has significantly improved accuracy by 10%, achieving an accuracy rate of 87.52% with FaceNet512 compared to previous methods. We also provide real-time performance readings at various stages of our approach and report promising inference times for MFR using the final model, which is as low as 0.024 seconds.