Automatic Deep Compression Based on Simplified Swarm Optimization

In recent years, convolutional neural networks (CNNs) have been proven and widely applied in the field of image recognition, including anomaly detection in manufacturing sites, and object detection in autonomous driving. However, the parameters obtained from the CNN increase exponentially with the depth of the network. Therefore, it is difficult to deploy the model in environments with limited computing resources. This study proposes a compression method for CNN by combining Simplified Swarm Optimization(SSO) with structured pruning. Our method can compress VGG16 to approximately 8.3 times smaller without sacrificing accuracy. The more important is, our method uses a heuristic approach to find the optimal pruning scheme without the need for repeated experimental verification.

I. Introduction

CNN has been one of the most successful applications in deep learning in recent years, with its effectiveness in processing large images, many industries have expected to deploy deep learning models on low-power and smaller edge devices. Typical edge devices have limited resources, with only a few hundred kilobytes to tens of megabytes of static memory, this results in complex models unable to be deployed easily. There are several methods for achieving model compression, but model pruning is a relatively easier approach to implement.