Test Case Prioritization Using Adaptive Random Sequence with Category-Partition-Based Distance

Test case prioritization schedules test cases in a certain order aiming to improve the effectiveness of regression testing. Random sequence is a basic and simple prioritization technique, while Adaptive Random Sequence (ARS) makes use of extra information to improve the diversity of random sequence. Some researchers have proposed prioritization techniques using ARS with white-box information, such as code coverage information, or with black-box information, such as string distances of the input data. In this paper, we propose new black-box test case prioritization techniques using ARS, and the diversity of test cases is assessed by category-partition-based distance. Our experimental studies show that these new techniques deliver higher fault-detection effectiveness than random prioritization, especially in the case of smaller ratio of failed test cases. In addition, in the comparison of different distance metrics, techniques with category-partition-based distance generally deliver better fault-detection effectiveness and efficiency, meanwhile in the comparison of different ordering algorithms, our ARS-based ordering algorithms usually have comparable fault-detection effectiveness but much lower computation overhead, and thus are much more cost-effective.

I. Introduction

A main task in regression testing is the prioritization of test cases, which reorders the existing test cases to provide cost-effective testing. Various test case prioritization techniques have been developed based on different information. For example, history-based prioritization techniques use information from previous executions to determine test priorities; knowledge-based techniques use human knowledge to determine test priorities; and model-based techniques use a model of the system to determine test priorities.