Normalized Cyclic Loop Network for Rail Surface Defect Detection Using Knowledge Distillation

In recent years, the application of computer vision for detecting rail defects has shown promising results. However, as the accuracy of the models improves, they become more complex with a large number of parameters, making it challenging to use them in practical scenarios. Although some lightweight models have been proposed to reduce the number of parameters, maintaining satisfactory performance remains difficult. Therefore, we propose a normalized cyclic loop network (NCLNet) using knowledge distillation (KD), called NCLNet-S*, for rail surface defect detection (RSDD). This model aims to be as lightweight as possible while preserving excellent accuracy. It achieves this by constructing an inter-modal feature cyclic enhancement structure to maximize the use of complementary features between modalities, incorporating a normalized fusion filter module for adaptive weighting of useful knowledge, and transferring knowledge from the teacher network (NCLNet-T) to the student network (NCLNet-S) using our proposed KD framework. Additionally, to enhance the performance and robustness of the NCLNet-S* model, we introduce intermodal cross-layer self-distillation. Extensive experimental results demonstrate that our proposed NCLNet-S* (NCLNet-S with KD) achieves high accuracy while remaining lightweight compared to state-of-the-art models. Furthermore, we conduct experiments on the publicly available RGBD-SOD datasets and achieve satisfactory performance, demonstrating the generality of our NCLNet-S*.