Multimodal Emotion Recognition in Deep Learning:a Survey

Emotion recognition is an active research issue in recent decades. It can be divided into implicit emotion recognition based on physiological signals and direct emotion recognition based on non-physiological signals. Direct emotion recognition through the processing of different modal information for emotion recognition is the emphasis of this paper. Firstly, we analyze the framework and research methods of video emotion recognition and music emotion recognition in detail, especially the research method based on deep learning. Then, the fusion technology in multimodal emotion recognition combining video and audio is compared. On the basis of the above contents, this paper reviews the research status of emotion recognition based on deep learning. Finally, according to the current research situation, we put forward some suggestions for future research.