Training Recurrent Neurocontrollers for Real-Time Applications

In this paper, we introduce a new approach to train recurrent neurocontrollers for real-time applications. We begin with training a recurrent neurocontroller for robustness on high-fidelity models of physical systems. For training, we use a recently developed derivative-free Kalman filter method which we enhance for controller training. After training, we fix weights of our recurrent neurocontroller and deploy it in an embedded environment. Then, we carry out additional training of the neurocontroller by adapting in real time its internal state (short-term memory), rather than its weights (long-term memory). Such real-time training is done with a new combination of simultaneous perturbation stochastic approximation (SPSA) and adaptive critic. Our critic is also a recurrent neural network (RNN), and it is trained by stochastic meta-descent (SMD) for increased efficiency. Our approach is applied to two important practical problems, electronic throttle control and hybrid electric vehicle control, with apparent performance improvement.

I. Introduction

Our research and that of collaborators have involved studies of neural networks (NNs) with internal feedback [5], [25], [6], [26]. Such NNs have been gaining popularity in the community due to their compactness and superior performance in dynamic system modeling and control. For example, Kim and Lewis [1] use adaptive networks with internal feedback as nonlinear observers for output feedback control of complex dynamic systems. This reference is an example of what may be called the traditional approach to neurocontrol, i.e., adaptation of network weights to cope with uncertainties.