Post-COVID-19 Mask-Aware Face Recognition System

Face recognition is a widely adopted technology in surveillance, border control, healthcare, banking services, and consumer product. Recent studies suggested that the performance of face recognition system is severely degraded in the presence of facial mask worn during COVID-19 pandemic. This work propose a mask-aware face recognition system that can identify subjects with and without facial mask presence. To this end, the two-fold contributions of this study are: (a) evaluation of the three hand-crafted descriptors, namely, Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), and Local Directional Order Pattern (LDOP) along with Support Vector Machine (SVM) for facial mask detection, and (b) deep learning based mask-aware dynamic ensemble model that can recognize subjects in facial mask presence and absence. Experimental evaluation are conducted on the Real-World Masked Face Recognition Dataset (RMFRD) consisting of 426 subjects with 1945 and 88500 images with and without facial mask, respectively. Results suggest highest accuracy of 99.60% in facial mask detection using LDOP-based descriptor. The performance degradation of up to 24% was reported for deep learning based ResNet-50 face recognition model in the presence of facial mask. Performance due to our proposed dynamic ensemble model (in the presence and absence of mask) is at par with the performance of the baseline face recognition system in the absence of facial mask. For instance, Equal Error Rate (EER) of the proposed dynamic ensemble (mask-aware face recognition) in the presence of facial mask is 6.59% and that of the deep-learning based face recognition system is 6.42% in the absence of facial mask.