Research of Convents Channel Pruning Based on MCP-L1 Regularization

Due to the advancement of mobile and wearable devices, large models with tens of billions or even trillions of parameters cannot be deployed on mobile devices, despite the excellent performance of high-performance GPUs. Therefore, compressing large models to operate efficiently on platforms with limited resources is a contemporary area of research. This study proposes a MCP-L1 filter reduction method based on the minimax concave penalty (MCP) function preprocessing to reduce model duplication and memory consumption. The new method improves the individual L1 penalty and is vulnerable to “outlier point” defects, while can execute feature selection with various levels of penalty for various parameters. For the ResNet34 model, under two different tailoring strategies, compared to the original model, the top 1 accuracy of MCP-L1 compressed model only decreased by 1.34% and 0.05%, but its floating-point calculation quantity has decreased by 26.63% and 15.62%. Furthermore, in all scenarios, when the pruning rate is the same, the MCP-L1 method has higher accuracy than L1-Norm method.