Considering Filter Importance and Irreplaceability In Filter Pruning

Deep convolutional neural network (CNNs) have gained a great success in computer vision tasks. However, the computation and parameter storage costs of CNNs are very large, thus a large number of studies have tried to reduce the computation and parameters of CNNs. Quantization and pruning are the usual strategies for model compression. Previous filter pruning works usually prune filters with small norm because the common viewpoint is that the smaller norm of filters, the less contribution of filters. However, above strategy ignores that features extracted by those filters with large norm may be redundancy and the features extracted by small norm filters are irreplaceable. In this paper, in light of above findings, we propose a novel filter pruning method called Importance Diversity Filter Pruning (IDFP), which considers both filter contribution and filter irreplaceability. We conduct extensive experiments on CIFAR-10 and CIFAR-100 datasets. The results illustrate the effectiveness of our method.