Self-Evolutionary Pose Distillation

Conventional pose distillation method utilizes the teacher's output and the label to co-supervise the student model. The architecture of the student model is fixed in the training, making it impossible to obtain a compact and powerful model. In this paper, we introduce the theory of evolution to the pose distillation and propose the self-evolutionary pose distillation (SEPD) method which not only improves the performance of the student model but also reduces the size of the student model. Specifically, the SEPD considers the original model as the teacher model and obtains the student model by using the model reduction strategy to shrink the teacher model. The student model is supervised by the teacher's output and the label jointly. After the optimization, the student is treated as the teacher model and the student model is obtained by the model reduction strategy. The student model is optimized again. The compact and strong model is obtained by repeating the procedure above. Experiments on the challenging benchmark validate the effectiveness of our SEPD method.