Pre-training Retnet of Simulating Entities and Relations as Sentences for Knowledge Graph Reasoning

Knowledge graphs have diverse applications, but a common challenge across these scenarios is the issue of missing triples. In this paper, we present a knowledge reasoning model based on the Retnet framework, employing a pre-training and fine-tuning approach. During the pre-training phase, we enhance the model's inference capabilities by replacing entities with sentence descriptions as inputs. This novel strategy aims to address the prevalent problem of missing triples in knowledge graphs. For various downstream tasks, we can fine-tune it to meet specific requirements. We conducted extensive experiments on several public datasets, covering triple classification, relationship prediction, and link prediction. The results consistently showcase the exceptional performance of our model. Particularly, in link prediction experiments on the FB15K-237 dataset, we achieved an impressive Mean Reciprocal Rank (MRR) of .356 and a Hits@10 score of 52.4.