Preventing Catastrophic Forgetting using Prior Transfer in Physics Informed Bayesian Neural Networks

Predictive models can be integrated in the sensing and monitoring methodologies of mechatronic systems in operation. When systems change or are subject to varying operating conditions, adaptivity of the models is needed. The goal of this paper is to enable this adaptivity by presenting a framework for continual learning. The framework aims to transfer and remember information from previously learned systems when a model is updated to new operating conditions. We achieve this by means of the following three key mechanisms. We first include physical information about the system, heavily regularizing the model output. Secondly, the usage of epistemic uncertainty, used as an indicator of the changing system, shows to what extend a transfer is desired. Last but not least the usage of a prior within a Bayesian framework allows to regularize models further according to previously obtained information. The last two principles are enabled thanks to the use of Bayesian neural networks. The methodology will be applied to a cam-follower system in a simulation environment, where results show that previously trained systems are better remembered with an increase of 72% compared to normal training procedures.