DeepFN: Towards Generalizable Facial Action Unit Recognition with Deep Face Normalization

Deployment of facial action unit recognition models has been impeded due to their limited generalization to unseen people and demographics. This work conducts an in-depth generalization analysis across several sources of variance: individuals (40 subjects), genders (male and female), skin types (darker and lighter), and databases (BP4D and DISFA). To help suppress the variance in data, we propose using self-supervised denoising autoencoders to transfer facial expressions of different people onto a common facial template which is then used to train and evaluate each of the models. We show that person-independent models yielded significantly lower performance (55% average F1 and accuracy across 40 subjects) than person-dependent models (60.3 %), leading to a generalization gap of 5.3%. However, normalizing the data with the proposed method significantly increased the performance of person-independent models (59.6%). Similarly, the proposed method was able to significantly reduce the generalization gap when considering gender (2.4%), skin type (5.3%), and dataset (9.4%). These findings represent an important step towards the creation of more generalizable facial action unit recognition systems.