Model predictive control with neural network system model and Bayesian last layer trust regions

Neural networks have proven to be an efficient and expressive black-box model to learn complex nonlinear relationships from large amounts of data. They are also increasingly popular in system identification for model predictive control (MPC). One of the main caveats of neural networks in this setting is their lack of uncertainty quantification. Unreasonable predictions in regions without training data might lead to poor and potentially dangerous control behavior. Bayesian neural networks try to alleviate this problem but add significant complexity to both training and inference, rendering their application for MPC infeasible. Bayesian last layer (BLL) is a simplified Bayesian neural network, representing a compromise between tractability and expressiveness. Most importantly, training and point estimates are unchanged in comparison to regular neural networks. The BLL covariance computation is strongly related to Gaussian Processes but in contrast to them, BLL does not suffer from the same scaling issues. While BLL cannot be used for probabilistic guarantees in most cases, we propose to define a trust region based on the computed covariance. We demonstrate in an empirical investigation that our economic MPC formulation with BLL trust region constraint leads to well behaved closed-loop trajectories, where the formulation without trust region leads to poor closed-loop performance.