Quantization Method Based on Gaussian Regularization Reduces Precision Loss in Reparameterization Quantization

As artificial intelligence rapidly advances, neural network models are becoming increasingly large and complex, significantly hindering their widespread application on edge devices. Research has shown that model compression techniques can reduce the memory usage and computational complexity of network models. The main methods of model compression include structural re-parameterization, knowledge distillation, and model quantization. Although combining two compression methods can benefit model deployment, simple combinations may lead to a collapse in inference accuracy. Therefore, improving inference accuracy after combining methods has become a focal point of research. To address the accuracy collapse issue caused by the combination of structural re-parameterization and model quantization, we propose a fine-tuning method using Gaussian regularization after training. Specifically, Gaussian regularization is applied during training to guide the distribution of re-parameterized model parameters towards a Gaussian distribution, thereby addressing the inference accuracy collapse from a post-training quantization perspective. Empirical results demonstrate that Gaussian regularization can reduce the accuracy error of models with both structural re-parameterization and quantization to within 1%.