Open-loop training of recurrent neural networks for nonlinear dynamical system identification

We develop a training approach for a class of recurrent neural networks which are categorized by layered links from input neurons to output neurons and time-lagged feedback links from output neurons to input neurons. This particular neural network structure can be considered as a special case of time-lagged recurrent networks. The present approach treats the recurrent neural network as a multilayer feedforward neural network during training by opening up the feedback links. We also treat the nonlinear system to be identified as a nonlinear function with no dynamics during data collection. Such a process for training data collection allows the use of random system states and random control inputs to ensure good representation in data collection and less dependence on the initial states. The training process of the neural networks can be simplified since the gradient calculation is much less involved in feedforward neural networks. It is argued that the neural network structure considered herein is appropriate for performing nonlinear dynamical system identification.

1 Introduction

Neural networks with at least one feedback loop are known as recurrent neural networks [5]. Recurrent neural networks have often been employed for nonlinear dynamic system identification and control [2], [6], [7], [9]–[11], [16]. There are many training approaches for recurrent neural networks reported in the literature (for a survey, see, e.g., [1], [8]).