Real-time Emotion Recognition and Expression for Avatar-Mediated Mental Health Counseling: Improving Nonverbal Communication and Emotional Response

Effective nonverbal communication is crucial in mental health counseling, and avatar-mediated communication offers potential solutions to enhance the nonverbal communication aspect. In this paper, we propose a real-time emotion recognition and expression model for avatar-mediated counseling and analyze its impact on the emotional experience and rapport building of clients. We recruited 60 college students, assigned 30 to a control group and 30 to an experimental group. The experimental group participated in avatar-mediated counseling sessions where the avatar responded with facial expressions that matched the emotions conveyed by the participants, while the control group received counseling sessions from an avatar without facial expressions. Our preliminary results showed that the experimental group perceived significantly higher sense of presence, immediacy, interactivity, and similarity with the avatar compared to the control group. Additionally, the experimental group reported significantly higher levels of satisfaction, trust, and rapport compared to the control group. Our study demonstrates the potential benefits of using avatars with real-time emotion recognition and expression in mental health counseling.