An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization

PLEASE REFER TO THE UPDATED VERSION OF THE REPLICATION PACKAGE. An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization This is the replication package associated with the paper "An Exploratory Study on the Impact of Change-proneness as a Metric in Black-box Test Suite Minimization". Replication Package Contents: This replication package contains all the necessary data and code required to reproduce the results reported in the paper. We provide the results of the Accuracy, Total Minimization Time (MT) for all the minimization budgets (i.e., 25%, 50%, and 75%). Data: We provide in the Data directory the data used in our experiments, which is the source code of test cases (Java test methods) of 16 projects collected from Defects4J. Code: We provide in the Code directory the code (Python) and bash files required to run the experiments and reproduce the results. Results: We provide in the Results directory the detailed results for our approach (called CTM). _________________________________ Get the dataset ready ---- Clones defects4j and prepares all the buggy versions of the dataset ---- cd Code/CTM bash init.sh Input: None Output: ./defects4j, ./defects4j-projects ------------------------------------------------------------------------------------------------------------------------ Extract Method Level Change Metrics ---- Extracts change metrics for all methods for each buggy versions of dataset ---- cd Code/CTM python3 extract_method_level_changes.py Input: ./defects4j-projects Output: Data/CTM/extracted_changed_methods ------------------------------------------------------------------------------------------------------------------------ Process Extracted Method Changes ---- Removes blank spaces, variable identifiers from function names ---- cd Code/CTM python3 process_extracted_method_names.py Input: Data/CTM/extracted_changed_methods Output: Data/CTM/extracted_changed_methods (updates the existing files) ------------------------------------------------------------------------------------------------------------------------ Merge Changes ---- Merges the duplicate methods ---- cd Code/CTM python3 merge_extracted_changes.py Input: Data/CTM/extracted_changed_methods Output: Data/CTM/merged_methods ------------------------------------------------------------------------------------------------------------------------ Process the call-graphs ---- Processes the callgraphs to match with the extracted metrics format ---- cd Code/CTM python3 process_callgraphs.py Input: Data/callgraphs Output: Data/CTM/processed_callgraphs ------------------------------------------------------------------------------------------------------------------------ Construct Test case - Method dependency mapping ---- Generates the multi-level dependency mapping of the test cases with methods ---- cd Code/CTM python3 construct_test_case_method_mapping.py Input: Data/processed_callgraphs Output: Data/CTM/method_invocations ------------------------------------------------------------------------------------------------------------------------ Calculate Change Proneness and Measure the Association with Test cases ---- Scores test cases based on their association with the change-prone methods ---- cd Code/CTM python3 calculate_test_case_change_proneness.py Input: Data/CTM/merged_methods, Data/CTM/unique_test_cases.csv, Data/CTM/method_invocations Output: Data/CTM/CP_values_of_test_cases ------------------------------------------------------------------------------------------------------------------------ Minimize ---- Selects the highest scoring test cases and performs minimization under three minimization budgets- 25%, 50% and 75% ---- cd Code/CTM python3 greedy_minimize.py Input: Data/CTM/CP_values_of_test_cases Output: Result/CTM/minimization_results ------------------------------------------------------------------------------------------------------------------------ Evaluate ---- Evaluates the accuracy of CTM configurations ---- cd Code/CTM python3 evaluate.py Input: Result/CTM/minimization_results Output: Result/CTM/accuracy ------------------------------------------------------------------------------------------------------------------------ Summarize the Project-wise Results ---- Generates project wise effectiveness accuracy summary ---- cd Code/CTM python3 summarize.py Input: Result/CTM/accuracy Output: Result/CTM/accuracy/summary