Local-Fusion Attention and Semantic-Augmentation Attention for Named Entity Recognition

Named Entity Recognition (NER) is one of the most fundamental and important tasks in Natural Language Processing. Recently, the self-attention mechanism has been widely applied in various models owing to its ability to capture longer contextual information. Numerous state-of-the-art approaches leverage self-attention or multi-head attention in conjunction with BiLSTM to further enhance NER performance. However, self-attention itself faces the challenge of insufficient sensitivity in extracting local features and additional semantic features, both crucial for improving NER. In this paper, we demonstrate that two structures, Local-Fusion Attention (LFA) and Semantic-Augmentation Attention (SAA), can effectively alleviate this problem. In particular, we propose Gate-Fusion module and Semantic-Fusion module to control information flow and balance input features, respectively. To evaluate our methods, we conducted extensive experiments on three benchmark datasets. The results show that our proposed model achieved improvements of 0.5%, 6.24%, and 3.02% over the baseline model on the CoNLL2003, WNUT2016, and WNUT2017 datasets, respectively. Furthermore, we compared our approach with some of the latest models and achieved the best results.