Artefact to our paper "An Empirical Study of Automated Unit Test Generation for Python"

<strong>Artefact for “An Empirical Study of Automated Unit Test Generation for Python”</strong> Together with our paper “An Empirical Study of Automated Unit Test Generation for Python”, we provide this artefact for future use. <strong>Pynguin Version</strong> We used Pynguin 0.25.2 for our experiments. The releases of Pynguin are archieved by Zenodo, too. Pynguin 0.25.2 is available under DOI 10.5281/zenodo.6836225. <strong>Preparation of the Environment</strong> We use the <code>poetry</code> dependency-management tool to manage all dependencies for this artefact. Install this tool if you have not done yet. Furthermore, let <code>poetry</code> create a virtual environment for the experiment by execution <code>poetry install</code>. <strong>Execution of the Experiment</strong> The execution scripts make several assumptions that are based on our infrastructure. We maintain a SLURM cluster infrastructure that defines different <code>constraints</code> for different machines. Furthermore, we assume some paths to be present: we assume every computing machine to have writable mount points at <code>/local/${USER}</code> and <code>/local/hdd/${USER}</code>. On our machines, both are mount points on the local hard disk/SSD of the computing machines. Additionally, we have a shared mount <code>/scratch/${USER}</code>, which is mounted via NFS from a central file server. This mount point is also mounted on all computing machines. We assume the created and packaged Docker image to be located at <code>/scratch/lukasczy/pynguin.tar</code>. You can change this path by editing the XML files. These XML files contain the basic definitions of the jobs: they specify the SLURM constraint, the version of the Pynguin Docker container, the used Pynguin configurations as well as the modules used for the experiments. These modules have to reside under <code>projects</code>, as they come with this artefact. The Python script <code>execution.py</code> generates the actual run scripts from the XML file. It generates all scripts necessary to run a SLURM array job consisting of all runs for the experiment. Further general settings for the SLURM array job are present in this file. The Bash script <code>run_experiment.sh</code> executes the full execution pipeline; one has to specify the variable <code>EXPERIMENT_NAME</code> to match the name of the respective XML file who's defined experiment shall be executed. <em>Important:</em> Executing the full experiment can take several days, depending on your computing infrastructure! We do therefore provide the raw result CSVs for further inspection. <strong>Data Analysis</strong> All raw data resides in the <code>data</code> folder: <code>loc_data.csv</code> contains all information about the lines of code in each module. This file was created using the <code>extract_locs_and_types.py</code> script in the root folder. Please note that executing this script requires that the <code>cloc</code> utility tool is installed on your system's path. <code>types.csv</code> and <code>types_per_module.csv</code> contain type information extract from the modules at different granularity level. They are also generated using the aforementioned script. <code>results-assertion.csv.xz</code> contains the raw results from the experiment for RQ3 that evaluates the effectiveness of the assertions. <code>results.csv.xz</code> contains the raw results from the experiment for RQ1 and RQ2. We provide the Jupyter Notebook that generated the plots, tables, and various LaTeX macros in the <code>notebooks</code> folder. Please note that if you want to reexecute this notebook, you might have to change the <code>PAPER_EXPORT_PATH</code> constant in cell <code>[2]</code> to a suitable location on your machine. Executing this notebook requires a installation of a TeX system to be available on your system because the plots are generated using <code>pdflatex</code> and <code>matplotlib</code>s pgf backend. <strong>Further Data</strong> The folder <code>projects</code> contains all the projects in the versions stated in our paper. The folder <code>run-logs</code> contains all the run logs from our experiment executions.