A Benchmark for SOTIF of Lane Marking Detection Algorithms of Autonomous Vehicles

Lane marking (LM) detection as an essential function for autonomous vehicles (AVs) has been extensively studied. Various machine learning-based algorithms are proposed to improve detection accuracy, while relatively less attention is paid to analyzing their performance limitations and quantifying their performance boundaries. Consequently, it becomes challenging to design a safe operational design domain where AVs are supposed to be operated safely. To address this issue, this paper proposes a benchmark for lane marking detection algorithms (LMDAs) to facilitate the determination of their performance boundaries. Specifically, we generated adversarial lane markings by automatically introducing random wear and regional wear to original challenging images selected from large-scale datasets. We also consider other factors such as image noise, brightness, contrast and color saturation for generating different types of adversarial lane markings. The resulting dataset with adversarial lane markings can be used to identify the performance boundaries of LMDAs. In evaluating the performance of LMDAs, the paper explores existing evaluation metrics and classifies them based on their application scopes. The results are of great value for guiding the development of LMDAs. In particular, the benchmark is useful for the Safety of the Intended Functionality (SOTIF) of LMDAs.