Brewing Up Reliability: Espresso Test Generation for Android Apps

The ESPRESSO testing framework for ANDROID has gained popularity among developers as it allows to write concise and reliable VI tests. State-of-the-art tools for automatically testing ANDROID apps, however, tend to produce crash reports rather than human-readable tests, and even if they produce tests these (1) rarely use the ESPRESSO format; (2) are often unreliable due to the volatile nature of widget identifiers; and (3) usually contain no test assertions to serve as regression oracles. While the lack of ESPRESSO support of test generation tools has been addressed by reverse engineering ESPRESSO tests, the other problems remain even with this workaround. In this paper, we therefore introduce a novel ESPREsso-based representation that allows test generators to generate ESPRESSO test cases directly that (1) can reliably identify widgets using clear and concise ESPRESSO selectors, and (2) can check test executions using ESPRESSO assertions. Experiments on 1,035 ANDROID apps demonstrate that the proposed approach generates ESPRESSO tests that are significantly more reliable than reverse engineered tests, and the ESPRESSO assertions of the generated tests are effective at detecting faults in ANDROID apps.