A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

This work presents a new sequential learning algorithm for radial basis function (RBF) networks referred to as generalized growing and pruning algorithm for RBF (GGAP-RBF). The paper first introduces the concept of significance for the hidden neurons and then uses it in the learning algorithm to realize parsimonious networks. The growing and pruning strategy of GGAP-RBF is based on linking the required learning accuracy with the significance of the nearest or intentionally added new neuron. Significance of a neuron is a measure of the average information content of that neuron. The GGAP-RBF algorithm can be used for any arbitrary sampling density for training samples and is derived from a rigorous statistical point of view. Simulation results for bench mark problems in the function approximation area show that the GGAP-RBF outperforms several other sequential learning algorithms in terms of learning speed, network size and generalization performance regardless of the sampling density function of the training data.

I. Introduction

Radial basis function (RBF) networks have gained much popularity in recent times due to their ability to approximate complex nonlinear mappings directly from the input–output data with a simple topological structure. Several learning algorithms have been proposed in the literature for training RBF networks [1] [2]–[12]. Selection of a learning algorithm for a particular application is critically dependent on its accuracy and speed. In practical online applications, sequential learning algorithms are generally preferred over batch learning algorithms as they do not require retraining whenever a new data is received. Compared with the batch learning algorithms, the sequential learning algorithms that we will discuss in this paper have the following distinguishing features: 1

all the training observations are sequentially (one-by-one) presented to the learning system;

2

at any time, only one training observation is seen and learned;

3

a training observation is discarded as soon as the learning procedure for that particular observation is completed;

4

the learning system has no prior knowledge as to how many total training observations will be presented.