FaceReZero Transformer for Unbiased Occlusion Invariant Deep Face Recognition with Train-Time Augmentations

The deep learning techniques for face recognition have achieved remarkable performance. However, the accuracy degrades when confronted with heavy occlusions like masks, goggles, scarves, and other obstructions. A further challenge is to recognize the low-quality images coming from the surveillance. This poses significant challenges for existing face recognition systems. To address these challenges, 1) we propose a set of 34 face data augmentation techniques tailored for real-world scenarios, designed to improve the accuracy of face recognition models. We do not recreate the training data after applying augmentations. Instead, we apply the augmentations directly to a mini-batch of 128 facial images during the training step, which adds only a mere 100 milliseconds overhead, regardless of the number of GPUs. It significantly enhances model accuracy across various mask face recognition challenge datasets while maintaining the accuracy on public benchmark datasets that do not include masks. Furthermore, 2) We adopted the ReZero is All You Need for face recognition and called it FaceReZero. It is a transformer-based model that combines overlapping patches, skip connections, an additional trainable parameter and a LayerNorm before Multi-head attention and MLP block in each layer. The FaceReZero demonstrated improved accuracy over Vision Transformer, Face Transformer and ReZero. In comprehensive evaluation, we assess the model trained with the proposed data augmentation and FaceReZero architecture on different test sets. Our approach achieved significantly improved accuracy on the MR-All, Children, and Multi-Racial test sets while demonstrating performance closely aligned with the Masks test set.

I. Introduction

In recent years, significant work has been made in the field of face recognition, owing to the availability of large-scale training datasets [1], [2], [3], sophisticated network architectures [4], [5], and advanced loss functions [6]. However, most existing face recognition systems primarily operate on non-occluded faces, where key facial features such as the nose, eyes, and mouth are unobstructed. The challenge intensifies when dealing with occluded faces due to various factors and traditional face recognition methods often struggle to effectively recognize occluded faces.