Contemporary Advances in Neural Network Quantization: A Survey

In the realm of deep learning, the advent of large-scale pre-trained models has significantly advanced computer vision and natural language processing. However, deploying these models on resource-constrained edge devices remains a significant challenge. Model quantization, a key technique in model compression and acceleration, offers a practical solution. This survey examines various quantization strategies, including symmetric versus asymmetric and uniform versus non-uniform approaches, and their implications for model performance. We particularly focus on methods addressing the precision loss inherent in quantization, thoroughly assessing Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ), as well as exploring Data-Free quantization and extreme quantization forms like binarization and ternarization. Additionally, we delve into recent advancements in quantizing specialized models such as Vision Transformers (ViTs) and Diffusion Models (DMs), highlighting the adaptability of quantization techniques. The survey concludes with a discussion of future research directions, with a focus on the potential of hardware co-optimization and no-data quantization strategies. This survey aims to inform and direct the adaptation of neural network models for efficient real-world deployment, serving as a comprehensive resource in the evolving field of model quantization.