Multimodal Emotion Recognition Based on Feature Fusion

In the field of human-computer interaction, human emotion recognition is a challenging problem, and it is also a key link to achieve barrier-free communication between human and machine. At present, most of the emotion recognition algorithms are constructed based on single modal social information, and the recognition results are one-sided and easily disturbed. The recognition accuracy is often difficult to meet the practical requirements after being separated from specific social environment conditions. Based on the above situation and problems, this paper adopts multimodal input and simultaneously includes three modal information of audio, text and facial expression to recognition emotion. Three single modal emotion recognition models are proposed based on three different input information, and the multimodal emotion recognition model are constructed by different feature fusion methods. The experimental results showed that the accuracy of multimodal model on the CH-SIMS dataset was 93.92%. In addition, compared with other emotion recognition models, the effectiveness of the proposed method is verified.