Improving generalization of radial basis function network with adaptive multi-objective particle swarm optimization

In this paper, an adaptive evolutionary multi-objective selection method of RBF Networks structure is discussed. The candidates of RBF Network structures are encoded into particles in Particle Swarm Optimization (PSO). These particles evolve toward Pareto-optimal front defined by several objective functions with model accuracy and complexity. The problem of unsupervised and supervised learning is discussed with Adaptive Multi-Objective PSO (AMOPSO). This study suggests an approach of RBF Network training through simultaneous optimization of architectures and weights with Adaptive PSO-based multi-objective algorithm. Our goal is to determine whether Adaptive Multi-objective PSO can train RBF Networks, and the performance is validated on accuracy and complexity. The experiments are conducted on two benchmark datasets obtained from the machine learning repository. The results show that our proposed method provides an effective means for training RBF Networks that is competitive with PSO-based multi-objective algorithm.

1. Introduction

Radial Basis Function (RBF) Networks form a class of Artificial Neural Networks (ANNs), which has certain advantages over other types of ANN s. It has three layers feed forward fully connected network, which uses RBFs as the only nonlinearity in the hidden layer neurons. The output layer has no nonlinearity and the connections of the output layer are only weighted, the connections from the input to the hidden layer are not weighted [1]. RBF Networks have been widely applied in many science and engineering fields. It is a feedback network of three layers, where each hidden unit implements a radial activation function and each output unit implements weighted sum of hidden units’ outputs.