Beyond back-propagation learning for diabetic detection: Convergence comparison of gradient descent, momentum and Adaptive Learning Rate

Back-Propagation (BP) algorithm is one of the classification algorithms in Artificial Neural Network (ANN). Back-Propagation still has problems with the training process, where learning convergence of gradient descent is very slow. Increase convergence rate can be done with the use of Adaptive Learning Rate and momentum optimization. Previous research that combining both optimization techniques was able to accelerate the training process. Therefore, this research will compare the convergence of gradient descent, gradient descent + momentum, gradient descent + adaptive learning rate and gradient descent + momentum + adaptive learning rate for diabetic detection. Based on the experimental results, it is known that gradient descent has better performance than others in terms of convergence velocity and result of TPR and FPR value. Training algorithm with combining of gradient descent + momentum + adaptive learning rate faster convergent compared to gradient + momentum or gradient + adaptive learning rate.