Convergence of Stochastic Gradient Decent Algorithm with Momentum

This paper aims to discuss the convergence of the stochastic gradient decent algorithm with momentum for feed-forward networks without hidden layers. Assuming that the required sample set is linearly independent, let all training samples be randomly grouped in each round of training, the number of samples in each group is fixed, and the weights are updated in the unit of group. The momentum and learning rate are selected in an adaptive manner, then we prove the convergence results of the learning process.