An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization

An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization

This is the replication package associated with the paper "An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization".

Replication Package Contents:

This replication package contains all the necessary data and code required to reproduce the results reported in the paper. We provide the results of the Accuracy, Total Minimization Time (MT) for all the minimization budgets (i.e., 25%, 50%, and 75%). The experiments were conducted using Python 3.12.3 and Java 1.8 on Ubuntu 24.04. Dataset Defects4j (version 2.0.1) was utilized. Please refer to Dataset Repository.

Data:

We provide in the Data directory the data used in our experiments, which is the source code of test cases (Java test methods) of 16 projects collected from Defects4J.

Code:

We provide in the Code directory the code (Python) and bash files required to run the experiments and reproduce the results.

Results:

We provide in the Results directory the detailed results for our approach (called CTM). _________________________________

Get the dataset ready

---- Clones defects4j and prepares all the buggy versions of the dataset ----

cd Code/CTM

bash init.sh

Input:

None

Output:

./defects4j

./defects4j-projects

------------------------------------------------------------------------------------------------------------------------

Extract Method Level Change Metrics

---- Extracts change metrics for all classes for each buggy versions of dataset ----

cd Code/CTM

python3 extract_change_metrics.py

Input: ./defects4j-projects

Output: Data/CTM/file_changes

------------------------------------------------------------------------------------------------------------------------

Merge Changes

---- Merges the duplicate classes ----

cd Code/CTM

python3 merge_class_changes.py

Input:

Data/CTM/file_changes

Output:

Data/CTM/merged_classes

------------------------------------------------------------------------------------------------------------------------

Construct Test case - Method dependency mapping

---- Generates the multi-level dependency mapping of the test cases with classes ----

cd Code/CTM

python3 construct_test_case_class_mapping.py

Input:

Data/callgraphs

Output:

Data/CTM/class_invocations

------------------------------------------------------------------------------------------------------------------------

Calculate Change Proneness and Measure the Association with Test cases

---- Scores test cases based on their association with the change-prone classes ----

cd Code/CTM

python3 calculate_test_case_change_proneness.py

Input:

Data/CTM/merged_classes

Data/CTM/unique_test_cases.csv

Data/CTM/class_invocations

Output:

Data/CTM/CP_values_of_test_cases

------------------------------------------------------------------------------------------------------------------------

Minimize

---- Selects the highest scoring test cases and performs minimization under three minimization budgets- 25%, 50% and 75% ----

cd Code/CTM

python3 greedy_minimize.py

Input:

Data/CTM/CP_values_of_test_cases

Output:

Result/CTM/minimization_results

------------------------------------------------------------------------------------------------------------------------

Evaluate

---- Evaluates the accuracy of CTM configurations ----

cd Code/CTM

python3 evaluate.py

Input:

Result/CTM/minimization_results

Output:

Result/CTM/accuracy

------------------------------------------------------------------------------------------------------------------------

Summarize the Project-wise Results

---- Generates project wise effectiveness accuracy summary ----

cd Code/CTM

python3 summarize.py

Input:

Result/CTM/accuracy

Output:

Result/CTM/accuracy/summary