Defects4J Workability for APR Results

Results for workability setup-test and trivial patch experiments on Defects4J version 2.0. setup-test-results.csv contains the result of the setup-test algorithm, executed 20 times on the full Defects4J dataset. The first rows contain meta-information on the runs: which of two different hosts for the VM was used, the parallelism setting, the aggregated (counted) results, and the total runtime (wall time) for the complete run. setup-test-run-*.json contain the raw output of the runs. The format is a list of objects for each bug in the dataset. Each such execution has the following fields: id executionIndex commandLine (list of the command-line to launch the tool) exitStatus (of the tool process) result (the result object, as documented in the README of the geneseer tool) logOutput started (timestamp) ended (timestamp) The result field can be mapped to the values seen in the csv file via the following jq command-line: jq -r '.[]|[if((.result.exception|type)=="string") then if(.result.exception|contains(" differs when run with coverage"))or(.result.exception|contains("Did not get single result when running test "))or(.result.exception|contains("Got duplicate test names in test suite result"))or(.result.exception|contains(" timed out when run with coverage")) then "INCONSISTENT_TEST_SUITE" elif(.result.exception|contains("net.ssehub.program_repair.geneseer.evaluation.CompilationException")) then "COMPILATION_ERROR" else "UNCLASSIFIED: "+.result.exception end else .result.result//.exitStatus end]|@csv' setup-test-without-classpath-fixes.csv contains the same as setup-test-results.csv information, but for a single run when not applying the custom fixes for the classpaths as exported by Defects4J. setup-test-without-classpath-fixes.json contains the raw output for this execution. geneseer-ease-rene-2026.zip contains the source code of the genseer tool. geneseer.jar contains the compiled tool with all dependencies included (geneseer-no-d4j-fixes.jar contains the compiled program with all dependencies included and the classpath fixes disabled). The file jgenprog-results.csv contains which defects in Defects4J 1.0 were reported to be fixed by jGenProg in M. Martinez and M. Monperrus, “Astor: Exploring the design space of generate-and-validate program repair beyond GenProg,” Journal of Systems and Software, vol. 151, pp. 65–80, May 2019, as well as whether they are also included in Defects4J 2.0. The file single-statement-deletion-results.csv contains the results of the experiment that deletes single statements to check for trivial fixes. The result of the experiment for each defect is in column 2: "FOUND_FIX" means that the test suite fully passes when deleting a single statement, "IMPROVED" that more test cases pass than originally, but not all, "NO_CHANGE" that no statement deletion can make more test cases pass. Additionally, the experiment was done with a 3 hour timeout for each defect; if that is reached, the result is "TIMEOUT". For all defects where a trivial fix is found ("FOUND_FIX" in column 2), column 3 shows the result of a manual investigation whether the original human-written patch for the defect contains only deletions. Column 4 shows which of the bugs is also in Defects4J version 1.0, and finally column 5 shows which bugs are fixed by jGenProg (see jgenprog-results.csv). single-statement-deletion-limit-100.json and single-statement-deletion-limit-300.json contain the raw output of the single-statement-deletion experiment, with deletion limits of 100 and 300 statements, respectively.