Holistic Feature Reconstruction-Based 3-D Attention Mechanism for Cross-Spectral Periocular Recognition

Recognition of individuals in cross-spectral environments involves situations where probe and gallery images are captured in distinct wavelength ranges and hence differ significantly in terms of appearance and illumination. In case of heterogeneous periocular images, alleviating this wide appearance gap and learning to extract illumination-invariant features from such local regions become cumbersome, giving rise to a challenging research problem. In this work, we design a novel holistic feature reconstruction-based attention module (H-FRAM) to refine and generate discriminative convolutional features. In contrast to existing spatial and channel attention mechanisms that compute 2-D or 1-D attention weights respectively, H-FRAM calculates the importance of each feature location by performing multi-linear principal component analysis on full 3-D tensor space. In H-FRAM, we claim that the feature reconstruction error, computed in a holistic manner, plays a crucial role in determining the relevance of feature locations. This reconstruction error is found by projecting the input feature map onto a multi-dimensional eigenspace that captures most of the important variations. To our knowledge, this is the first work which explores subspace learning approach in the context of 3-D attention mechanism to capture discriminative feature information. We have created an in-house cross-spectral periocular dataset containing visible and near-infrared images from 200 classes. The images are captured in unconstrained acquisition setup involving unsupervised eye and head movements as well as accessory variations (face masks and eyeglasses). Extensive experiments and ablation studies show that the proposed network achieves state-of-the-art recognition performances on the existing and in-house datasets for heterogeneous and homogeneous periocular recognition as well as heterogeneous face recognition.