Hierarchical Multi-layer Transfer Learning Model for Biomedical Question Answering

Biomedical semantic question answering refers to answering questions from a given contextual passage in the biomedical field. The traditional methods need complicated feature engineering and the end-to-end neural network models depend on the large scale dataset. We propose a hierarchical multi-layer transfer learning model to address the question answering task on the biomedical data which lacks of sufficient training data. The domain adaptation techniques are adopted to reinforce the performance, include fine-tuning and forgetting cost regularization which penalize the deviations from the source model's parameters and avoid forgetting the knowledge of source domain. The distributed representation of the word is generated and domain knowledge of biomedical word embedding is integrated. Especially, the co-attention mechanism captures the question interaction clues for passage encoding. The open data set of 2017 BioASQ-Task 5B is used to evaluate the system performance. The results show that the domain adaptation techniques make the system get the state-of-the-art performance. Our model without any handcrafted feature achieves higher precision than the best solution for factoid question in 2017 BioASQ-Task 5B.