Fake News Detection Based on Cross-modal Co-attention

The spread of rumors has brought many negative impacts to society. Nowadays, rumors on social media platforms often exist in the form of both images and text. In response to this, many methods for multimodal rumor detection have been proposed, but most of these methods only utilize extracted text features and image features for simple fusion, without considering the connections between modalities. In this study, we employ ConvNet to extract image features, BERT to extract text features, and take into account the complementarity between modalities. We use a co-attention module to enhance the extracted features of individual modalities, aiming to achieve the interaction of modality features and the complementary nature between modality information. The obtained features are then concatenated to achieve the fusion of image and text features. The multimodal fusion model used in this study was experimented on Twitter and Weibo, achieving accuracy rates of 81.20% and 88.15% respectively.