A Survey on Application of Knowledge Distillation in Healthcare Domain

Deep neural networks have become increasingly popular in both industry and academia due to the availability of vast amounts of data and computing resources. Transferlearning can be effective in cases where obtaining a large training dataset is not possible, allowing complex models trained on large datasets to be fine-tuned for specific tasks. However, the healthcare sector faces challenges due to the lack of readily available data and resources, and the complexity and memory requirements of pretrained models can make them difficult to deploy. To address these issues, knowledge distillation has been widely used in healthcare to compress large and complex models, making them easier to deploy. This study provides a comprehensive review of recent research that has utilized knowledge distillation to build deep learning models for various tasks in healthcare, as well as identifying potential areas for future research.