Survey and Experiments on Biomedical Pre-Trained Language Models for Named Entity Recognition

The use of language models and deep learning has arisen as promising solution for Named Entity Recognition in biomedicine due to the outperforming results in the last years. However, several challenges remain under research, such as training under different conditions to detect biomedical entities in texts, and to choose the best model for each entity. Besides, since the number of detected genetic variations is growing, the recognition of mutation entities is highly demanded. In the present work, we fine-tuned and compared the performance of publicly available relevant pre-trained language models under the same condition to detect biomedical entities. In addition, we applied an extra fine-tuning step, and observed an outstanding F1 improvement for mutation entities.