Replication Package for the paper "AI-based Fault-proneness Metrics for Source Code Changes"

This is the replication package for the paper "AI-based Fault-proneness Metrics for Source Code Changes", submitted at the IWSM-Mensura '23 conference.

The archive is a Docker image file with a fully setup and working environment to re-execute the experiments involved in the manuscript. We pre-loaded all libraries and codeBERT models to ease the replication process and avoid compatibility issues, as the environment cannot be easily managed using Dockerfiles.

To run the image, a Docker installation is needed. Once downloaded, from the command line type:

docker load -i

After the loading process, you can run the container by typing:

docker run -it mensura/ai-proneness-replication:1.0

All the source code and the dataset to re-execute the experiment is located into the /Replication folder. The folder contains the results of our experimentation in CSV and MS Excel format, along with the following subdirectories:

dataset: a replication of the used dataset. The file dataset.csv gives information on all the entries, while the code folder contains a subdirectory for each sample, named by its id. In the folder, the file old.txt and new.txt refers to the older and newer version of the method, respectively; gitdiff.txt stores the raw git-diff command output, while diff.html stores a more human-readable version of the differences.

ai-fault-proneness-tk-replication: the Java code used to apply Tree Kernel techniques on the dataset (we used JDK-11, embedded within the container). To build and execute the package, refer to the file README.md in the folder. For convenience, we also provided an executable JAR file ai-fault-proneness-tk-replication-1.0-jar-with-dependencies.jar that can be run directly and saves the output in a CSV file in the results folder of the replication package.

code-embeddings-and-analysis: python scripts to execute the codeBERT-based approaches and to extract the diff statistics. To execute all the steps, a convenience shell script execute.sh has been pre-loaded and can be executed to automatize all the process.