Model Compression Based on Knowledge Distillation and Its Application in HRRP

In High Resolution Range Profile (HRRP) target recognition approach, ultra-deep network, huge amount of parameter, and the time and space consumption for model deployment are the problems of conventional one-dimensional Convolutional Neural Network (CNN). To solve these problems, a model compression method based on knowledge distillation is proposed in this paper. In this method, a high accurate complex network is trained firstly, and then a conventional cross-entropy loss function and a loss function based on softening processing of output value is combined to build and train a simple network. Finally a lightweight and accurate recognition model is obtained. The experimental results of actual dataset show that the method proposed in this paper not only reduces the parameter amount and calculation consumption of the lightweight model greatly, but also approaches the recognition performance of the complex network.