CNDA: Chinese Multi-label Classification Model based on Dual Attention

The introduction of Bidirectional Encoder Representations from Transformers (BERT) represents a pivotal juncture in the evolution of natural language processing. BERT-based neural network models are widely used in text classification due to their proven effectiveness. However, existing multi-label classifiers utilizing BERT often neglect the impact of word pairs on overall text semantics. To address this limitation, Chinese multi-label classification model based on dual attention (CNDA) is proposed in this paper. Employing attention mechanisms during the encoding phase facilitates the sentence to capture the influence of each individual word. The bidirectional processing, in turn, equips the model with the capability to apprehend long-range dependencies and contextual nuances. Within the classifier, the integration of an attention-based conditional random field (CRF) offers the capacity to dynamically adjust the feature function weights through attention weights. This adaptability allows the model to accommodate diverse input sequences, thus enhancing the accuracy and generalization prowess of sequence labeling. We evaluated the performance of the proposed model on several classic datasets. The result indicates the proposed model effectively captures the impact of word pairs on text semantics and outperforms existing models in terms of accuracy.