FPGA-Based CNN Quantization Method and Hardware Acceleration Module

In the field of AI inference chips, quantization can effectively reduce the parameter size and computational cost. Additionally, the design of hardware acceleration modules can significantly accelerate the inference speed and reduce the resource consumption caused by computation and parameter storage. According to the mainstream neural network parameter structure, this paper presents a post-training quantization method. The optimal base point, which are composed of the sum of powers of 2, is determined by searching according to the Euclidean distance. Fixed-point mapping is then carried out to achieve quantization of different bit widths. Simultaneously, the matrix calculation acceleration unit is designed on the field-programmable gate array (FPGA) platform using the characteristic of power of 2 multiplication calculation. This work replaces the original multiplication calculation with shifting, significantly reducing hardware resource consumption. Experimental results demonstrate that when weight values and feature maps are quantized to 4-bits, the model compresses to 14.3% of its original size with only a 2.4% loss in network accuracy. The module for accelerating quantitative inference runs at 300MHz on the Xilinx XC7Z100 platform, resulting in a 339 times faster inference speed compared to the ARM platform.