Self-Distillated DETR for Object Detection in Complex Environments

Detection transformers (DETRs) have achieved remarkable results, but they still face challenges in generalizing to complex environments. Existing approaches primarily emphasize specific computational structural design but overlook the overall improvement of encoder structure. To improve the detection accuracy of DETR in complex environments, we propose a self-distillated DETR, which enhances encoder structural feature extraction through self-distillation. First, we develop an encoder feature self-distillation architecture, which performs deep-layer to shallow-layer knowledge transfer in the encoder structure without external instructor guidance. Second, we design the feature distillation loss function, which guides shallow encoders to acquire deep encoder feature extraction capability. We use the smoothed gradient squared deviation to calculate the corresponding token feature differences. We conduct experiments using the public object detection dataset MS COCO and the natural wildlife dataset WAID to verify the model's accuracy. The results of the experiment show the proposed model's effectiveness in multiple precision metrics.