An Efficient Second-Order Algorithm for Self-Organizing Fuzzy Neural Networks

Intelligent computing technologies are useful and important for online data modeling, where system dynamics may be nonstationary with some uncertainties. In this paper, an efficient learning mechanism is developed for building self-organizing fuzzy neural networks (SOFNNs), where a second-order algorithm (SOA) with adaptive learning rate is employed, the network size and the parameters can be determined simultaneously in the learning process. First, all parameters of SOFNN are adjusted by using the SOA strategy to achieve fast convergence through a powerful search scheme. Second, the structure of SOFNN can be self-organized using the relative importance index of each rule. The fuzzy rules used in SOFNN with SOA (SOA-SOFNN) are generated or pruned automatically to reduce the computational complexity and potentially improve the generalization power. Finally, a theoretical analysis on the learning convergence of the proposed SOA-SOFNN is given to show the computational efficiency. To demonstrate the merits of our proposed approach for data modeling, several benchmark datasets, and a real world application associated with nonlinear systems modeling problems are examined with comparisons against other existing methods. The results indicate that our proposed SOA-SOFNN performs favorably in terms of both learning speed and prediction accuracy for online data modeling.

I. Introduction

In the past decades, it is well known that fuzzy neural network (FNN) has been widely used in multiple domain areas including modeling and control of the nonlinear systems, image recognition, and other industrial processes [1]–[4]. This is due to the ability of FNNs to integrate the advantages of both neural networks and fuzzy systems with learning ability and interpretability [5]–[7]. It has been shown that FNNs can approximate any continuous nonlinear maps defined over a compact set with arbitrary accuracy [8], [9]. However, the established universal approximate theorem does not tell us how to properly construct an FNN using a given dataset, in particular, a streaming dataset with varying dynamics for time being. This means, for some applications, FNNs have technical limits and need further developments on adaptation aspects, such as structure update techniques and effective learning schemes [10], [11].