IntegrationDistiller: Automating Integration Analysis and Testing of Object-Oriented Applications

Software systems typically consist of various interacting components and units. While these components can be tested and shown to work correctly in isolation, when integrated and start interacting with each other, they may fail to produce the desired behaviors and results. Integration testing plays an important role in revealing issues in interactions among cooperating components. Identifying different interaction scenarios, however, is not a trivial task when performing integration testing. On the other hand, most of the integration testing solutions proposed in the literature are manual which hinders their scalability and applicability when it comes to large industrial systems. In this paper we introduce IntegrationDistiller as an automated solution and tool to identify integration scenarios and generate test cases (in the form of method call sequences) for .NET applications. It works by analyzing the code and automatically identifying class couplings, interacting methods, as well as invocation points. Moreover, the tool also helps and supports testers in identifying timing issues at integration level by automatic code instrumentation at invocation points. The code analysis engine of IntegrationDistiller is built and automated using .NET compiler platform, known as Roslyn. Hence, this work is the first in utilizing Roslyn features for automatic integration analysis and integration test case generation. This work has been done as part of our collaboration with ABB Industrial Automation Control Technologies (IACT) in Västerås-Sweden to address the integration testing challenges of the software part of the ABB Ability™ 800xA distributed control systems.

I. Introduction

With the transition of traditional industrial systems towards more software-based solutions, the role of software in industry has become more and more dominant and important than ever before. Such transitions are already observed in domains such as automotive, telecommunication, and process industry as more functionality and features are implemented and assigned to software components in a product. On the other hand, this transition necessitates more rigorous verification and validation as the quality of the embedded software can now have direct and bigger impact on the quality of the end-product that is delivered to the customers (e.g., a car). This also brings along the following requirements on the testing techniques for ensuring the quality of software products in industry: •

scalability: the testing techniques adopted should be scalable with respect to the increasing size and complexity of real-world industrial software applications (e.g., amount of code committed, and changes made daily by different development teams located in different parts of the world; as in distributed development environments),

•

automation: as the size and complexity of industrial software products grow rapidly, automation in testing gains an important role particularly related to the scalability issue as manual solutions can be too costly and time-consuming to scale well and be sustainable in the long run. This is particularly important considering the frequency of code commits and daily changes made by different distributed development teams as in continuous integration and deployment environments, and when continuous decisions on test prioritization, selection and execution for regression testing need to be made repeatedly,

•

seamless integration: testing techniques should not disrupt and negatively impact the current development process of a company, and should fit seamlessly with it. There are many proposed testing techniques in the literature which may perform better than current ones already used in a company, but are not adopted simply due to this issue.