Quantized ADAM with Monotonically Increasing Resolution of Quantization

We propose a quantized ADAM learning algorithm that can represent better optimization performance by monotonically reducing the quantization step to time when quantization is composed of integer or fixed-point fractional values applied to an optimization algorithm. According to the White Noise Hypothesis to quantization error with dense and uniform distribution, we can regard the quantization error as an i.i.d. white noise. It leads that we can obtain a stochastic equation about the quantized ADAM learning equation, and we obtain the monotonically decreasing rate of the quantization step that enables the global optimization by the stochastic analysis to the derivation of an objective function. Numerical experiments represent that the proposed algorithm represents a better performance than the conventional ADAM learning schemes on a ResNet for a general image classification test.