Accurate Modeling of Performance Histories for Evolving Software Systems

Learning from the history of a software system's performance behavior does not only help discovering and locating performance bugs, but also identifying evolutionary performance patterns and general trends, such as when technical debt accumulates. Exhaustive regression testing is usually impractical, because rigorous performance benchmarking requires executing a realistic workload per revision, which results in large execution times. In this paper, we propose a novel active revision sampling approach, which aims at tracking and understanding a system's performance history by approximating the performance behavior of a software system across all of its revisions. In a nutshell, we iteratively sample and measure the performance of specific revisions that help us building an exact performance-evolution model, and we use Gaussian Process models to assess in which revision ranges our model is most uncertain with the goal to sample further revisions for measurement. We have conducted an empirical analysis of the evolutionary performance behavior modeled as a time series of the histories of six real-world software systems. Our evaluation demonstrates that Gaussian Process models are able to accurately estimate the performance-evolution history of real-world software systems with only few measurements and to reveal interesting behaviors and trends.