Expresso-AI: An Explainable Video-Based Deep Learning Models for Depression Diagnosis

Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention. As a multidisciplinary topic, these objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. Even though current automated depression diagnosis approaches improved over the last decade, a critical gap exists as they often lack affect-specificity and interpretability, limiting their practical application and potential impact on mental health care. In particular, interpretability from temporal activities from videos when deep models are used is not fully explored. In this study, we present a novel framework for analyzing Deep Neural Networks’ decisions when trained on facial videos, specifically focusing on automatic depression severity diagnosis. By fine-tuning Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets on depression severity facial videos from AVEC depression dataset, our framework is able to interpret the model’s saliency maps by examining face regions and temporal expression semantics. Our approach generates both visual and quantitative explanations for the model’s decisions, providing greater insight into its reasoning. In addition to this interpretability, our video-based modeling has improved upon previous single-face benchmarks for visual depression diagnosis, resulting in enhanced predictive performance. Overall, our work demonstrates the successful development of a framework capable of generating hypotheses from a facial model’s decisions while simultaneously improving depression’s predictive capabilities.

I. Introduction

Depression is a pervasive mental health issue affecting millions of people globally. According to the World Health Organization (WHO), over 264 million people of all ages suffer from depression, making it one of the leading causes of disability worldwide [1]. Current diagnostic methods for depression primarily rely on subjective assessments, such as self-reports and clinical interviews, which may be influenced by factors like personal bias, cultural differences, and subjective interpretations. Given the widespread prevalence of depression and its consequential impact on individuals and society, there is a pressing need for developing objective measures to accurately identify and assess depressive symptoms. Such objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. By addressing this need, we can pave the way for improved early diagnosis, intervention, and ultimately, better mental health outcomes for those affected by depression.