A hybrid of cuckoo search and simplex method for fuzzy neural network training

In this paper, a new hybrid algorithm mixing the simplex method of Nelder and Mead (NM) and the cuckoo search (CS), abbreviated as NM-CS, is proposed for the training of the Fuzzy Neural Networks (FNNs). In standard CS, cuckoo birds engage the obligate brood parasitism by laying their own eggs to other host birds. If a host bird discovers the alien eggs, they will either throw these eggs away or abandon its nest and build a new nest elsewhere. In the proposed hybrid algorithm, instead of using the probability to discover an alien egg for the CS, we use the concept of a simplex which is used in the NM algorithm to abandon and generate the new nests. Our proposed method puts more emphasis on exploration of the search space and enhances the ability to avoid local optimum. Some simulation problems will be provided to compare the performances of the proposed method and other methods in training an FNN. In these simulations, it is observed that the proposed method outperforms other methods.

I. Introduction

Fuzzy neural networks (FNNs) have long considered successful learning machines in terms of function approximation, pattern recognition, classification, and image processing. They can be used for problem solving if no known mathematical models of a given problem are available. The universal approximation property is important for the success of an FNN in a variety of applications [1], [4]. This provides more flexibility in designing an appropriate learning machine for nonlinear problems. The advantage of using fuzzy neural network for machine learning is that its parameters usually have clear physical meanings and we have some intuitive methods to choose good initial values for them. There exist many algorithms for training an FNN such as back-propagation algorithm (BP), genetic algorithm (GA) [5], particle swarm optimization algorithm (PSO) [6], differential evolution (DE) [7] and so on. The simplest FNN learning construction makes use of the incremental gradient descent approach in BP algorithm [8]. Unfortunately, the approximating solution using BP algorithm may easily get trapped in local minima of the cost surface, especially for those non-linearly separable pattern classification problems or complex function approximation problems, and never finds a near-optimal solution [9]. Moreover, it is quite sensitive to the initial settings of the connection weights and learning rate.