Efficient Dynamic Fixed-Point Quantization of CNN Inference Accelerators for Edge Devices

This paper presents a dynamic fixed-point quantization flow. Different bit-width configurations of features, weights, and partial sums can be evaluated comprehensively over the state-of-the-art convolutional neural networks. The proposed quantization scheme simplifies the hardware design for on-device inference with the help of our analysis framework. The experiment results show that the energy consumption can be improved by up to 29.55% with little accuracy degradation by representing features/weights of 8 bits, and partial sums of 16 bits.