A Tucker Decomposition Based on Adaptive Genetic Algorithm for Efficient Deep Model Compression

Model compression is a prevalent technique to efficiently deploy CNNs on resource-constrained devices, which show outstanding performance in computer vision. Recent efforts devote to approximately representing the higher-order tensors in CNNs without hurting original accuracy. However, the selection of ranks for various decomposition algorithms is still an open problem. We propose an efficient model compression method which leverage Tucker decomposition for global compression. An improved genetic algorithm is applied to selecting and optimizing the ranks for Tucker decomposition. The improved genetic algorithm can also actively detect and get rid of local optimal, which is main drawback of search algorithms. Different decomposition strategies for convolutional layer and fully connected layer are designed based on their dimensional difference. Extensive experiments show that with negligible loss of accuracy, proposed method gives a compression ratio of 7.57× in number of parameters and 2.69× in FLOPs to VGG-19 on CIFAR-100, which has advanced the state-of-the-art methods.