Enhancing Cyber-Physical System Analysis with Structure-Aware Modular Neural Networks

In a wide array of engineering disciplines, modular designs serve as a fundamental approach for constructing intricate systems from simpler constituent modules. Domain-specific knowledge about the resulting structures can be formalized, paving the way for informed machine learning, a methodology that leverages prior knowledge to augment data-driven techniques with symbolic insights. One of many possible ways to include prior knowledge into a machine learning pipeline is through incorporation into the design of neural network architectures. We propose a novel algorithm for the learning of modular neural networks on modular cyber-physical systems where structural knowledge is used to build network architectures which resemble technical modules and the information flow is based on semantic relations between data and modules. We investigate potential benefits of modular neural networks compared to classical monolithic approaches based on anomaly detection experiments on data sets of mechanical pendulums with variable numbers of joints which are provided with this research. We define and investigate different levels of knowledge integration and modularity and show that modular designs can have a positive impact on anomaly detection performance. All models and code are published as open source.