Metamorphic Testing Group Generation and Prioritization Technique Based on Path Analysis

Metamorphic testing (MT) leverages metamorphic properties (usually known as metamorphic relationships, MR) of the software under test (SUT) to generate the follow-up test cases from the source test cases, and verify the test results by examining whether the MR is followed by the corresponding outputs. MT effectively alleviates the oracle problem because it is no longer necessary to obtain the expected outputs of individual test cases. Obviously, the used MRs and source test cases play a key role in the fault detection effectiveness of MT. Although there are already some test case generation methods for MT, they have the following limitations. Firstly, the scope of applicable input domains for an MR is not carefully considered, which may result in invalid test cases. Secondly, only the differences between source test cases are considered during the test case generation, which may result in insufficient metamorphic testing groups (a pair of source test case and follow-up test case, briefly as MTG). Finally, the fault detection capability of test cases is not considered, which may affect the fault detection efficiency of MT. In order to solve the above limitations, we propose a metamorphic testing group generation and prioritization technique based on path analysis (PaMTG). PaMTG first obtains all path pairs of the MR through analyzing the possible paths of the SUT, then generates MTGs to cover as many path pairs as possible, and finally prioritizes the derived MTGs according to their covered path information. A supporting tool was developed and an empirical study was conducted to evaluate PaMTG in terms of valid MTG ratio, fault detection capability, fault detection rate, and time overhead. The experimental results show that PaMTG is able to generate valid MTGs, and the fault detection capability and fault detection rate of the generated MTGs are better than that of the existing baseline techniques.