Algorithm or Representation? An empirical study on how SAPIENZ achieves coverage

Testing is a very important and expensive part of developing Android applications. Several tools for automatically testing Android applications have been proposed. In particular, Sapienz is a search-based tool that has been recently deployed in an industrial setting. Although it has been shown that Sapienz outperforms several state-of-the-art tools, it is still to be seen what features of Sapienz impact the most on its effectiveness.We conducted an extensive empirical study where we compare the impact of the search algorithm and the usage of motif genes, a more compact representation of individuals. Our empirical study shows that the usage of motif genes improves statement coverage both for evolutionary algorithms and random approaches. In particular, our study shows that although the evolutionary algorithm used by Sapienz (i.e., NSGA-II) outperforms other search algorithms, it is not statistically distinguishable from Random Search. These facts cast doubts about the use of evolutionary algorithms in the context of Android test generation and suggest that motif genes have a great impact on the overall effectiveness.

1 Introduction

As software keeps becoming more important in our daily lives, the use of mobile devices such as smartphones and tablets increases as well. It is estimated that mobile technologies are now used by two-thirds of the global population. Furthermore, mobile users universally consume more digital minutes per person – more than double in the vast majority of countries and regions [2] . In this context, smartphones have become the dominant platform for mobile time consumption, in terms of total minutes across every market. About 80% of all mobile time [2] is spent in application consumption (commonly known as “apps”). As of January 2020, there are over 2.8 million applications available on Google’s Play App Store [4] .