Using Sensitivity Analysis and Search-Based Testing in the Verification of a Computer Vision Function

For operating safely, autonomous robots and automated driving applications need to reliably detect objects in their environment under a huge variety of external conditions. Previous works introduced the usage of semantic domain models for capturing important domain parameters that have an impact on detection performance of perception systems. In these approaches, rating the importance of domain parameters is usually left to the intuition and experience of a test engineer. In this paper, we present a framework utilizing global sensitivity analysis (GSA) for determining the impact of domain parameters on detection performance and exploiting search-based testing (SBT) technology for generating difficult input images for testing a computer vision function based on the domain parameters identified by GSA. In addition, we discuss the benefits and lessons learned of the interplay of GSA and SBT for testing a perception function. In our evaluation, we apply our framework on a pedestrian detection use case from the automated driving domain. Our evaluation results show that SBT finds critical test images faster than random testing and that selecting the right domain parameters based on GSA is crucial for good test results.