Exploring Emotion and Sentiment Landscape of Depression: A Multimodal Analysis Approach

Conventionally depression detection was done through extensive clinical interviews, wherein the subject’s responses are studied by the psychologist to determine his/her mental state. In our model, we try to imbibe the sentiment analysis and emotion detection approach by capturing video and analyzing facial and audio input to identify the symptoms of depression. The system is built with a video capturing containing an emotion recognition, utilizing Haar Cascade classifier, CNN models, and an audio processing model using Google API, NLP forwarded with Naïve Bayes Classifier. We have used FER 2013 dataset. The multimodal approach gives almost 77% accuracy, which can be integrated with the self-assessment questionnaire to improve the accuracy of depression detection. Many depressed individuals cannot receive the right care since there are few objective ways to assess depression. The results of this model with minor modifications will be incorporated into the system for early detection along with intervention for depression and anxiety amongst youth well-being in our society.