Simulating optical properties to access novel metrological parameter ranges and the impact of different model approximations

The evolving market of advanced driver assistance and autonomous driving is expected to reduce road fatalities and disrupt mobility. Many of such systems sense the environment using visual range-cameras, detecting road signs, pedestrians and cars to support the driving functions with reliable data. To ensure the reliability and functional safety, extensive testing is required, which is costly, time consuming and sometimes even not feasible or dangerous. Validation with the beneficial help of simulation has therefore advanced in recent years allowing for direct testing of critical or costly scenarios, which are hard to test in real world. However, this requires both accurate enough models and a low sampling overhead by using suitable model approximations. Following established real camera measurement processes, such as ISO12233 or similar, simulation itself needs to be quantified, certified and testing limits need to be specified. In this article we apply a physical-realistic lens simulation to an existing image database, simulating an image as if it were seen with a different lens. The changes are quantified using the metrological metric Modulation Transfer Function (MTF) and other numerical metrics (Structural Similarity Index, Mean Squared Error). We then run an object detection algorithm on this degraded imaged database. Additionally, these findings are compared to standard neural network performance metrics (Average Precision, AP). Whereas the Deep neural network is sensitive to the optical model itself (drop in AP by 6%) it turns out that it is robust to the small perturbations introduced by different approximations. Interestingly, this is in contrast to the MTF values, where we see a distinct dependence on the level of approximation. This article may serve as starting point to answer the question of how well does the optical model in simulation need to be sampled? How many details do really matter?