Multi-Feature Fusion Transformer for Natural Language Inference

Natural Language Inference (NLI) is a branch of Natural Language Processing (NLP) whose main task is to determine the relationship between two sentences. Such tasks essentially use pre-trained models to ensure accuracy, and many applications are found in human-computer dialog and question-answering systems. However, in some cases where small devices are deployed offline, lightweight model implementations are often required to save computational resources. To address this problem and ensure more efficient inference capabilities, this paper describes how to improve the preexisting Transformer and proposes a Multi-Feature Fusion Transformer Network for NLI processing. The model integrates sequence features and introduces a priori information to enhance local features. It also achieves a comprehensive fusion of sequence information, local features, and non-local features, thus compensating for many potential shortcomings of Transformer. At the same time, it maintains the lightweight feature of the model and facilitates the implementation of downstream tasks. We verified the above point by experiments, our model has better inference performance compared to previous models used for the same task. The accuracy in the SNLI dataset reached 89.0%, while the matched and mismatched versions of MultiNLI reached 79.3% and 78.7%.