Applying metamorphic testing to reliability calculating programs

Reliability evaluation is imperative for developing safety-critical systems where failures can cause serious consequences. Thus, assuring the correctness of programs that compute reliability is indispensable. However, this is a challenging task when the correct output is not known a prior, since it is impossible to determine whether the calculated reliability is correct or not. To overcome this challenge, we propose the application of metamorphic testing to reliability calculating programs. Metamorphic testing tests software by checking whether particular relations, generally called metamorphic relations, hold or not between multiple inputs and outputs rather than solely focusing on the correctness of independent outputs. Therefore, this testing approach allows software to be tested without known correct outputs. In this study, we focus on programs that compute the reliability of coherent systems from a given set of pathsets. We developed several metamorphic relations, which allowed us to automatically test two programs. To the best of our knowledge, this is the first study where metamorphic testing was applied to reliability calculating software.