Distilling the Long-Range Knowledge for Real-Time Semantic Segmentation

Deep convolutional networks have achieved great success in semantic segmentation. However, existing semantic segmentation networks are difficult to achieve an ideal balance between accuracy and speed. To tackle this dilemma, the key idea of this paper is to design a two-part knowledge distillation framework to improve the accuracy of real-time networks, including separated distillation and long-range distillation. Separated distillation treats semantic segmentation as an independent pixel classification task, aiming to align logits between a cumbersome teacher network and a lightweight student network. To extract the long-range knowledge from the scene, the similarity attention map is defined as knowledge, which is transfered from teacher to student through the long-range distillation. The framework proposed in this paper improves the accuracy of real-time networks on Cityscapes and Fisheye datasets by 2.7% and 1.7%, respectively. The distilled network is tested on NVIDIA Xavier embedded platform with speed of 15FPS. Code will be available here: https://github.com/WangYunnan/Real-Time-Semantic-Segmentation.