Multi-modal emotion recognition based on gated composite network

In the process of multimodal sentiment analysis, there is a need to mine each modality with deeper semantic information, and this paper proposes a lightweight deep learning architecture for gated composite networks. The gating mechanism is used to model the local interactions between different modalities, and the Dynamic Time Domain Convolutional Network (DTCN) is used to efficiently model the time-domain relationships within the feature sequences and to mitigate the effects of redundant information on the inter-modal interactions. In the experimental tests, the proposed algorithm outperforms the comparative algorithms in both regression and classification tasks, which proves that it has a more desirable comprehensive performance.