Image and Text Emotion Classification Model Based on Multitask Learning

In bimodal emotion analysis, if the interaction between modes and the different contributions of each mode to the final emotion classification result are not considered in emotion classification, it will lead to the problem of low final classification accuracy. Firstly, to solve the problem of easily losing feature location information during pooling, the relative order of feature values is preserved. We chose the k-max pooling strategy and proposed BERT combined with TextCNN model to generate high-level text features. Then, for the problem of poor correlation between feature channels, using an improved SE ResNeXt residual network to obtain image features improves training efficiency and optimizes the performance of the model. Thirdly, aiming at the problem of overly focusing on interactive information while ignoring the independence of each mode in bimodal emotional fusion, the interactive information between modes and the internal information of modes is combined through symmetric attention-based cross-modal interaction and gated network. Finally, the advantages of multitask learning, additional related tasks are added through parameter sharing to improve the accuracy of emotional classification and generalization of the model. Experimental results show that our method is better in accuracy and efficiency than the state of the art method.