Fusing Local-Global Facial Features by NFFT for Automatic Depression Estimation

With the increasing number of depressed patients and the development of computer vision technology, the study of individual automatic depression estimation (ADE) methods based on facial images has attracted much attention in recent years. Most existing works focus on obtaining informative features from the whole images with advancing deep learning models, while the local tiny changes and the fusion of different features have been paid less attention. In this paper, a two-branch predicting model with a elaborate transfomer block NFFT is designed to combine global and local features extracted from whole images and image patches for predicting the depression score precisely. Besides, a classification head is added to guide the regression results for improving accuracy. Experiment results on the AVEC2014 dataset (MAE=5.81, RMSE=7.49) demonstrate that the proposed model outperforms other methods, and the extended experiments on one new dataset (CCPL) are conducted to validate the generalizability and robustness of our model.