Modeling Cyber Physical Systems with Learning Enabled Components using Hybrid Predicate Transition Nets

Cyber-physical systems (CPSs) are ubiquitous ranging from smart household appliances to drones and self-driving cars, and are becoming increasingly important in the functioning of our society. In recent years, learning enabled components (LECs) built using machine learning approaches are increasingly used in CPSs to perform autonomous tasks to deal with uncertain and unfamiliar environments. In this paper, an approach for formally modeling CPSs with LECs is presented. Hybrid predicate transition nets are used to model LECs built using deep neural nets and reinforcement learning. Specifically, a method for modeling deep neural nets and their training using hybrid predicate transition nets is developed. Additionally, generic hybrid predicate transition net structures are designed to model reinforcement learning based on neural fitted Q-learning. The expressive power of hybrid predicate transition nets supports all commonly used activation and cost/reward functions in deep neural nets and reinforcement learning. The operational semantics of hybrid predicate transition nets enables the online and offline training of deep neural nets as well as online and offline policy update in reinforcement learning. Furthermore, hybrid predicate transition nets are used to model the overall CPS with LECs through the Simplex architecture. These results (1) provide an executable symbolic representation combining logic and algebraic definitions for two major machine learning approaches, and (2) contribute a systematic and unified framework to study CPSs with LECs. The modeling method is demonstrated using a vehicle benchmark problem.