On the Energy Footprint of Mobile Testing Frameworks

High energy consumption is a challenging issue that an ever increasing number of mobile applications face today. However, energy consumption is being tested in an
ad hoc
way, despite being an important non-functional requirement of an application. Such limitation becomes particularly disconcerting during software testing: on the one hand, developers do not really know how to measure energy; on the other hand, there is no knowledge as to what is the energy overhead imposed by the testing framework. In this paper, as we evaluate eight popular mobile UI automation frameworks, we have discovered that there are automation frameworks that increase energy consumption up to roughly 2200 percent. While limited in the interactions one can do,
Espresso
is the most energy efficient framework. However, depending on the needs of the tester,
Appium
,
Monkeyrunner
, or
UIAutomator
are good alternatives. In practice, results show that deciding which is the most suitable framework is vital. We provide a decision tree to help developers make an educated decision on which framework suits best their testing needs.