One-shot Hierarchical Global Pruning via Channel Relation-aware Attention

Network pruning is an effective way of model compression, which has achieved significant research progress in image classification. However, existing pruning methods primarily evaluate the importance of the channel feature itself, and then perform multiple iterative pruning based on a global pruning ratio. These methods ignore the interdependence among channels within the same layer and the iterative pruning is very time-consuming. In this paper, we propose a One-shot Hierarchical Global Pruning based on channel relation-aware attention, dubbed as OHGPruner, which introduces a Channel Relation-aware Attention(CRA) to model the interdependence among channels, and measures channel importance in terms of two dimensions: original features and relation features. In addition, we also introduce a hierarchical global strategy, which improves the shortcomings of global pruning while avoiding the "layer collapse" issue caused by one-shot pruning. Our method only needs to prune once, which significantly improves pruning efficiency while maintaining the accuracy of the pruned model. Extensive experiments are conducted on two public datasets (CIFAR-10 and ILSVRC-2012) for different models. The experimental results show that OHGPruner achieves higher accuracy with similar FLOPs and Parameters as other methods.