“Sampling” as a Baseline Optimizer for Search-Based Software Engineering

Increasingly, Software Engineering (SE) researchers use search-based optimization techniques to solve SE problems with multiple conflicting objectives. These techniques often apply CPU-intensive evolutionary algorithms to explore generations of mutations to a population of candidate solutions. An alternative approach, proposed in this paper, is to start with a very large population and sample down to just the better solutions. We call this method “Sway”, short for “the sampling way”. This paper compares Sway versus state-of-the-art search-based SE tools using seven models: five software product line models; and two other software process control models (concerned with project management, effort estimation, and selection of requirements) during incremental agile development. For these models, the experiments of this paper show that Sway is competitive with corresponding state-of-the-art evolutionary algorithms while requiring orders of magnitude fewer evaluations. Considering the simplicity and effectiveness of Sway, we, therefore, propose this approach as a baseline method for search-based software engineering models, especially for models that are very slow to execute.

1 Introduction

Software engineers often need to answer questions that explore trade-offs between competing goals. For example:1)

What is the smallest set of test cases that cover all program branches?

2)

What is the set of requirements that balances software development cost and customer satisfaction?

3)

What sequence of refactoring steps take the least effort while most decreasing the future maintenance costs of a system?