Zero-Keep Filter Pruning for Energy Efficient Deep Neural Network

Recent deep learning models succeed to achieve high accuracy and fast inference time, but they require high-performance computing resources because of a large number of parameters. However, not all systems have high-performance hardware. Sometimes, deep learning model needs to be run on edge devices such as IoT devices or smartphones. The edge devices have limited performance and energy consumption. On these devices, the amount of computation must be reduced. Pruning is one of the well-known approaches to solve this problem. In this work, we propose "zero-keep filter pruning" for an energy-efficient deep neural network. The proposed method maximizes the number of zero elements in filters by replacing small values with zero and pruning the filter that has the lowest number of zeros. In the conventional approach, the filters that have the highest number of zeros are generally pruned. As a result, through this zero-keep filter pruning, we can have the filters that have many zeros in a model. We compared the results of the proposed method with the random filter pruning and proved that our method shows better performance with much fewer non-zero elements with marginal accuracy drop. We also compare the number of remained filters with random and proposed pruning methods after pruning. Finally, we discuss a possible multiplier architecture, zero-skip multiplier circuit, which skips the multiplications with zero to accelerate and reduce energy consumption.