Beyond SOTIF: Black Swans and Formal Methods

The ISO 26262 standard addresses system failures and the need to mitigate them safely. However, the standard is only implicit regarding the safety of the intended functionality. One should concede that a system without failures, operating in the specified design boundaries should be safe. None the less, the new ISO/PAS 21448 standard on Safety of the Intended Functionality (SOTIF) only explicitly addresses unintended functionality as it pertains to the design of the product, purportedly in the absence of any system, element or component failures. How can this have happened? What guarantees that contemporary complex computer steered systems always behave well under normal circumstances without showing any unexpected and deviant behavior that can be potentially hazardous to the user? How can this conundrum be amended? This paper explores the actual reality of failures in complex systems that rely on complex sub-systems to produce the desired functionality. We challenge the notion that the ISO 26262 and the ISO 21488 standards are presently sufficient in its guidance to resolve this enigma.