Enhancing Representations Based on Inherent Information

Pre-trained models have achieved remarkable success. However, existing methods have not fully utilized the inherent information. To address such limitation, we propose to enhance representations based on inherent information. To fuse more inherent information into our model, we propose a graph which contains rich information. To utilize inherent information, we further design three tasks.