Learning Label Independence and Relevance for Multi-Label Biomedical Text Classification

The rapidly growing biomedical literature requires accurate and robust automatic computational methods to quickly select the most relevant labels in biomedical topic candidate label sets to specific biomedical documents. Such methods can facilitate hypothesis generation and knowledge discovery. Many multi-label biomedical text classification methods have been proposed in the last few years, such as DeepMeSH, MeSHProbeNet, and BERTMeSH. However, these methods encode the labels as one-hot vectors, which ignore the semantic relevance between the labels. Moreover, the loss function they employ does not consider the unbalanced label distribution, resulting in overfitting labels with high frequencies. To alleviate the above problems and improve the performance of multi-label biomedical document classification tasks, we propose a model to learn Label Independence And Relevance (LIAR) and integrate them efficiently. LIAR uses BioBERT to fully extract information from biomedical literature to generate textual representations and uses it to construct a one-hot vector and learn label embeddings, respectively. Meanwhile, we construct a new loss function that can adaptively weight and integrates the one-hot vector distribution and label semantic similarity to compute the loss value and Assign Weights (AWLoss) to labels of different frequencies to alleviate the shortcomings of the loss function in the above model. LIAR outperformed the state-of-the-art method by more than 1% on all three benchmark datasets.