DeepTest: Using Machine Learning for Generating new Testsequences

When performing functional tests, stress tests or runtime tests of embedded systems, a large amount of data is collected. The tests are partly performed by developers and system integrators in suitable test environments, but also in acceptance tests or field trials. These test runs have usually been evaluated, albeit not comprehensively, by testers and/or customers, and can thus serve as a reference for further tests.Here we describe, how we trained neural networks using this data, with the goal of generating and executing new tests.In doing so, we take two approaches. On the one hand, we look at the test runs as time series of signal values, the input and output values over time steps during the test runs. On the other hand, we look at sliding time windows over these runs, similar to oscilloscope images, and analyze these signal images.The quality of generated data, at least as for generated image data, is mostly evaluated visually. This is not sufficient for the intended scope of testing. For this reason, we introduce, for the first time according to our knowledge, quantitative meaures to evaluate the quality of generated signal traces.We show how such an extension can be systematically integrated into a development process.