Universal approximation with error bounds for dynamic artificial neural network models: A tutorial and some new results

The three main classes of dynamic artificial neural network models for identification of nonlinear dynamical systems are reviewed: (1) neural state-space models, (2) global input-output models, (3) dynamic recurrent neural network models. The presentation of the mathematical models and architectures are followed by their representations in terms of a consistent block diagram convenient for stability and performance analyses and argued to potentially have benefits for process identification. The classes of nonlinear dynamical systems that are universally approximated by such models are characterized, with rigorous upper bounds on the approximation errors. While many of the results are available in the literature, this paper is the first to fully develop and clearly explain these models and their interrelationships to provide a broader perspective, and presents some new results to fill in the gaps in the literature.