Adaptive parallel identification of dynamical systems by adaptive recurrent neural networks

Under mild regularity conditions, a dynamical system can be approximated to any accuracy by a recurrent neural networks (NN) [J. T. Lo, July 1993]. This universal approximation property qualifies recurrent NNs as system identifiers in the parallel formulations. If a dynamical system under identification is affected by an uncertain environmental parameter, online adjustment of the weights of the system identifier is necessary to adapt to the environmental parameter in the parallel formulation. However, adjusting all the weights of a recurrent NN involves much online computation, poor local minima to fall into, long transient states, and even divergence of the output of the recurrent NN. This motivated and development of adaptive multilayer perceptrons with longand short-term memories (i.e. MLPWINs with long- and short-term memories), which are intended to eliminate all these difficulties. Mathematical justification of adaptive MLPWINs for parallel system identification was reported in (J. T. Lo et al., July 1999). Numerical feasibility of the same studied. Simple dynamical systems selected from three classes of dynamical systems, namely systems with nonlinear actuation, bilinear systems, and systems with bifurcation and chaos, are used in our study. In all three examples, it is shown that adaptive MLPWINs property trained as adaptive parallel identifiers adapt effectively to changing environmental parameters even with values not included in the a priori offline training data.

1. Introduction

It is proven in [2] that a dynamical system with an uncertain environmental parameter can be approximated, to any accuracy under mild regularity conditions, by an adaptive multilayer perceptron with interconnected neurons (MLPWIN). which is an MLPWIN with with long- and short-term memories, the long-term memory being independent of the environmental parameter. This allows us to determine the long-term memory in an a priori offline training and adjust only the short-term memory online to adapt to the uncertain environmental parameter. Since the short-term memory consists of linear weights of the adaptive MLPWIN, online adjustment can be performed by any fast LMS or RLS algorithm for adaptive linear filtering. This has the online benefits of less computation, no convergence to poor local minima, and shorter transients as compared with a recurrent NN with all their weights adjusted online.