Contrastive Learning-Based Multi-Modal Entity Alignment and Feature Fusion Method for Electric Power Knowledge Graph

This paper proposes a Contrastive Learning-Based Multi-Modal Entity Alignment and Feature Fusion Method (CMEAF) for addressing the challenge of poor entity alignment in Chinese electric power knowledge graph retrieval tasks. Firstly, we combine the ideas of multi-modal contrastive learning from CLIP and momentum contrastive learning from MoCo to design an improved contrastive loss function (MCCL). By introducing a more flexible negative sample selection strategy and incorporating momentum updates, the performance of contrastive learning is enhanced. Secondly, in the multi-modal feature fusion stage, we propose a Cross-Modal Attention Fusion (CMAF) method that leverages cross-modal attention mechanisms to fully exploit the interactive information between image and text features, achieving superior multi-modal feature fusion. Lastly, in the model pre-processing stage, we adopt the Mixup data augmentation idea and propose a Joint Data Augmentation approach based on image-text pairs (TMIXUP) to enhance downstream retrieval performance. Experimental results demonstrate significant performance improvements of our proposed method in multi-modal entity alignment and feature fusion tasks. Compared to the baseline model WUKONG, CMEAF achieves a 2.5% increase in average recall rate on the MUGE dataset and a 4.1% improvement in MR performance on the Flickr30k-CN dataset. On the self-built POWER dataset, CMEAF also achieves excellent retrieval performance with an MR score of 93.7%. Furthermore, this paper conducts visual analysis to further showcase the advantages of multi-modal fusion. The proposed method effectively aligns Chinese cross-modal entities with high accuracy and practicality.