Data of Differential Testing of Concurrent Classes - ICST 2025

ConDiff Artifact - ICST 2025 Differential Testing of Concurrent Classes Valerio Terragni University of Auckland, New Zealand [e-mail]://valerio-terragni.github.io/ Shing-Chi Cheung The Hong Kong University of Science and Technology, Hong Kong Accepted at ICST 2025 @inproceedings{Terragni2025ConDiff, author = {Valerio Terragni and Shing-Chi Cheung}, title = {Differential Testing of Concurrent Classes}, booktitle = {The 18th IEEE International Conference on Software Testing, Verification and Validation (ICST)}, year = {2025}, publisher = {IEEE},} Subjects and Results Folders Overview The following folders contain all the program revisions used in our experiments: - benchmarksC-B - benchmarksC-BR100 - benchmarksC-BR50 - benchmarksC-BR25 Each folder includes two program versions: - V1: The version without the fault. - V2: The faulty version. Results Folder The `results` folder contains the outputs of ConDiff and the Baseline. The files are organized as follows: - Files starting with `true_`: Results of ConDiff with the filtering phase enabled. - Files starting with `false_`: Results of the Baseline, i.e., the variant of ConDiff without the filtering phase enabled. Each file includes the results of 10 runs for each subject. --- Results Details The results include the detected behavioral differences (if any) and the following metrics (all time measurements are in milliseconds): - SR: Success Rate - DRT: Difference Revealing Time - CT: Number of concurrent tests generated - CTB: Number of concurrent tests analyzed by the behavioral checker - CTPRUNED: Number of pruned tests - PREFIXPRUNED: Number of pruned prefixes that lead to sequential behavioral differences - SUFFIXPRUNED: Number of pruned suffixes that lead to sequential behavioral differences - CFP: Number of concurrent function pairs containing at least one changed method Time Measurements - TIME_SCHANGE: Time to identify methods that are changed across revisions - TIME_GEN: Time to generate the tests - TIME_DIFFSEQ: Time for sequential behavioral checking - TIME_DIFFCON: Time for concurrent behavioral checking - TIME_CIA: Time for change impact analysis