Simulink Compiler Testing via Configuration Diversification With Reinforcement Learning

Simulink compiler testing is important since all cyber-physical system (CPS) models are required to be compiled by Simulink compiler. Current testing processes use CPS models generated by CPS model generators for testing. Since the effectiveness of CPS model generators heavily relies on suitable generator configurations, existing approaches randomize configurations or infer configurations with historical bug information to generate diverse bug-triggering CPS models. However, these approaches are designed for general-purpose compilers (e.g., GCC), which have two challenges when testing Simulink compiler, namely, the CPS model representation challenge on representing CPS models for diversity measurement and the configuration learning challenge on learning configurations to generate diverse CPS models. To address these challenges, we propose
R
einforcement l
E
arning-based
CO
nfigu
R
ation
D
iversification (RECORD), a new configuration diversification approach. RECORD has a feature vectorization component, which addresses the first challenge by representing CPS models as feature vectors to capture the local and global characteristics of CPS models for diversity measurement. RECORD then uses a reinforcement learning component to generate diverse CPS models based on the learned relationship between configuration updates and diversity changes, thus addressing the second challenge. Experiments demonstrate that within three months, RECORD reported 11 confirmed Simulink compiler bugs, significantly outperforming the state-of-the-art configuration diversification approaches. RECORD can also facilitate different testing strategies to find more bugs.