Chaos Driven Development for Software Robustness Enhancement

The importance of software quality increases as software products become more intertwined with our daily lives. One critical software quality attribute is software robustness reflecting the confidence level that the software can still function correctly against stressful environment and invalid inputs. To build up a robust software system, we need start from the chaos to understand the system behavior, and intentionally create the chaos condition to identify the potential gaps. In this paper, we analyze the main root causes leading to software robustness issues and get them classified into two main categories. Starting from the real issues, we raise the proposal for chaos driven validation, debuggability and development process enhancements. It dramatically reduces the new robustness issues on Intel Media Driver by 10 times and the average debug time by 6 times in 1.5 years. We hope the experiences shared in this paper can help more practitioners to apply similar enhancements.

I. Introduction

Along with our software scale, it becomes more and more distributed and ephemeral which poses higher requirements on software quality. Software Robustness is widely accepted as one key factor of software quality. It reflects the confidence level that software can still function correctly (or at least return elegantly) against stressful environment or invalid inputs. To promise the software robustness, we need start from the chaos to understand the system behavior, and intentionally create the chaos condition to proactively identify the potential gaps.