Novel direct and self-regulating approaches to determine optimum growing multi-experts network structure

This work presents two novel approaches to determine optimum growing multi-experts network (GMN) structure. The first method called direct method deals with expertise domain and levels in connection with local experts. The growing neural gas (GNG) algorithm is used to cluster the local experts. The concept of error distribution is used to apportion error among the local experts. After reaching the specified size of the network, redundant experts removal algorithm is invoked to prune the size of the network based on the ranking of the experts. However, GMN is not ergonomic due to too many network control parameters. Therefore, a self-regulating GMN (SGMN) algorithm is proposed. SGMN adopts self-adaptive learning rates for gradient-descent learning rules. In addition, SGMN adopts a more rigorous clustering method called fully self-organized simplified adaptive resonance theory in a modified form. Experimental results show SGMN obtains comparative or even better performance than GMN in four benchmark examples, with reduced sensitivity to learning parameters setting. Moreover, both GMN and SGMN outperform the other neural networks and statistical models. The efficacy of SGMN is further justified in three industrial applications and a control problem. It provides consistent results besides holding out a profound potential and promise for building a novel type of nonlinear model consisting of several local linear models.

I. Introduction

NEURAL NETWORKS are capable of learning and reconstructing complex nonlinear mappings and have been widely studied by control researchers in the identification and design of control systems. The hypothesis space may be thought of as containing all control actions that an artificial neural network with a given architecture may, in principle, learn. Until recent years, a conventional artificial neural network would have a static architecture, i.e., the number of neurons and their connectivity patterns would be specified and fixed by the network's designer prior to training the network. The static structure has actually restricted the space of hypotheses that it can compute. In designing the network architecture, the possible control action space for certain control applications has to be taken into consideration. However, in most control problems, the perturbations acting on the system and the appropriate control actions to execute are not known a priori. Therefore, it is not possible to predetermine the network structure. In order to overcome the limitations introduced by a priori fixed hypothesis space, the neural networks should have to allow for modifications of their structure as a function of learning. In fact, many of the limitations encountered by neural networks are owing to a fixed architecture. It has been shown that training a fixed size network cannot be solved in polynomial time [42], even when it has only three hidden nodes [41]. This suggests fundamentally different learning properties of neural network models that allow structural changes during learning.