Optimum learning rate for backpropagation neural networks

Backpropagation (BP) is a systematic method for training multilayer artificial neural networks (NNs). Although it has dramatically expanded the range of problems to which NNs can be applied, BP networks suffer from slow learning mainly due to a constant, non-optimum learning rate (a fixed step size) /spl eta/. In this paper, an optimum, time-varying learning rate for multilayer BP networks is analytically derived. Results show that training time can be reduced significantly while not causing any oscillations during the training process.<
>