Artefact to our paper "An Empirical Study of Automated Unit Test Generation for Python"

<strong>Artefact for “An Empirical Study of Automated Unit Test Generation for Python”</strong> Together with our paper “An Empirical Study of Automated Unit Test Generation for Python”, we provide this artefact for future use. <strong>Pynguin Version</strong> We used Pynguin 0.25.2 for our experiments. The releases of Pynguin are archieved by Zenodo, too. Pynguin 0.25.2 is available under DOI 10.5281/zenodo.6836225. <strong>Preparation of the Environment</strong> We use the <code>poetry</code> dependency-management tool to manage all dependencies for this artefact. Install this tool if you have not done yet. Furthermore, let <code>poetry</code> create a virtual environment for the experiment by execution <code>poetry install</code>. <strong>Execution of the Experiment</strong> The execution scripts make several assumptions that are based on our infrastructure. We maintain a SLURM cluster infrastructure that defines different <code>constraints</code> for different machines. Furthermore, we assume some paths to be present: we assume every computing machine to have writable mount points at <code>/local/${USER}</code> and <code>/local/hdd/${USER}</code>. On our machines, both are mount points on the local hard disk/SSD of the computing machines. Additionally, we have a shared mount <code>/scratch/${USER}</code>, which is mounted via NFS from a central file server. This mount point is also mounted on all computing machines. We assume the created and packaged Docker image to be located at <code>/scratch/lukasczy/pynguin.tar</code>. You can change this path by editing the XML files. These XML files contain the basic definitions of the jobs: they specify the SLURM constraint, the version of the Pynguin Docker container, the used Pynguin configurations as well as the modules used for the experiments. These modules have to reside under <code>projects</code>, as they come with this artefact. The Python script <code>execution.py</code> generates the actual run scripts from the XML file. It generates all scripts necessary to run a SLURM array job consisting of all runs for the experiment. Further general settings for the SLURM array job are present in this file. The Bash script <code>run_experiment.sh</code> executes the full execution pipeline; one has to specify the variable <code>EXPERIMENT_NAME</code> to match the name of the respective XML file who's defined experiment shall be executed. <em>Important:</em> Executing the full experiment can take several days, depending on your computing infrastructure! We do therefore provide the raw result CSVs for further inspection. <strong>Data Analysis</strong> All raw data resides in the <code>data</code> folder: <code>loc_data.csv</code> contains all information about the lines of code in each module. This file was created using the <code>extract_locs_and_types.py</code> script in the root folder. Please note that executing this script requires that the <code>cloc</code> utility tool is installed on your system's path. <code>types.csv</code> and <code>types_per_module.csv</code> contain type information extract from the modules at different granularity level. They are also generated using the aforementioned script. <code>results-assertion.csv.xz</code> contains the raw results from the experiment for RQ3 that evaluates the effectiveness of the assertions. <code>results.csv.xz</code> contains the raw results from the experiment for RQ1 and RQ2. We provide the Jupyter Notebook that generated the plots, tables, and various LaTeX macros in the <code>notebooks</code> folder. Please note that if you want to reexecute this notebook, you might have to change the <code>PAPER_EXPORT_PATH</code> constant in cell <code>[2]</code> to a suitable location on your machine. Executing this notebook requires a installation of a TeX system to be available on your system because the plots are generated using <code>pdflatex</code> and <code>matplotlib</code>s pgf backend. <strong>Further Data</strong> The folder <code>projects</code> contains all the projects in the versions stated in our paper. The folder <code>run-logs</code> contains all the run logs from our experiment executions.

### 《Python自动化单元测试生成实证研究》配套工件
本配套工件配合《Python自动化单元测试生成实证研究》论文发布，可供后续研究者复用。

#### Pynguin版本
本次实验使用Pynguin 0.25.2版本。Pynguin的各发行版均由Zenodo存档，Pynguin 0.25.2的存档DOI为10.5281/zenodo.6836225。

#### 环境准备
本研究工件的所有依赖项通过<code>poetry</code>依赖管理工具进行管理。若尚未安装该工具，请先完成安装。随后执行<code>poetry install</code>命令，由<code>poetry</code>为实验创建独立虚拟环境。

#### 实验执行
实验执行脚本基于本团队的基础设施环境设定了若干预设条件。本团队维护了一套SLURM集群基础设施，可为不同计算节点配置不同的<code>节点约束</code>。此外，脚本同时预设了若干目录路径：假设每台计算节点均拥有可读写的挂载点<code>/local/${USER}</code>与<code>/local/hdd/${USER}</code>，在本团队的硬件环境中，这两个路径均指向计算节点本地硬盘/固态硬盘的挂载目录。同时，本团队还拥有共享挂载目录<code>/scratch/${USER}</code>，该目录通过中央文件服务器的<code>网络文件系统（Network File System, NFS）</code>实现挂载，且所有计算节点均已挂载该目录。脚本预设打包完成的Docker镜像存放路径为<code>/scratch/lukasczy/pynguin.tar</code>，用户可通过编辑XML配置文件修改该路径。这些XML文件包含了作业的基础定义：用于指定SLURM节点约束、Pynguin Docker容器版本、所用Pynguin配置参数以及实验所用的待测项目。所有待测项目需存放于<code>projects</code>目录下，随本工件一同分发。Python脚本<code>execution.py</code>可从XML配置文件生成实际的运行脚本，生成用于执行本次实验全部任务的SLURM阵列作业所需的所有脚本，该配置文件中还包含了SLURM阵列作业的其他通用设置项。Bash脚本<code>run_experiment.sh</code>用于执行完整的实验执行流程，用户需将变量<code>EXPERIMENT_NAME</code>设置为待执行实验对应的XML配置文件名，以匹配实验定义。<em>重要提示：</em>根据您的计算基础设施规模，完整执行本次实验可能耗时数日。因此，本工件同步提供了实验原始结果CSV文件，供用户直接检视。

#### 数据分析
所有原始数据均存放于<code>data</code>目录下：<code>loc_data.csv</code>包含了各待测项目的代码行统计信息，该文件由根目录下的<code>extract_locs_and_types.py</code>脚本生成。请注意，运行该脚本需确保系统PATH中已安装<code>cloc</code>工具。<code>types.csv</code>与<code>types_per_module.csv</code>包含了以不同粒度从待测项目中提取的类型标注信息，同样通过上述脚本生成。<code>results-assertion.csv.xz</code>包含了针对研究问题3（RQ3，评估断言生成有效性）的实验原始结果；<code>results.csv.xz</code>则包含了针对研究问题1（RQ1）与研究问题2（RQ2）的实验原始结果。本工件在<code>notebooks</code>目录中提供了用于生成论文图表、表格及各类LaTeX宏的Jupyter Notebook文件。请注意，若需重新运行该Notebook，您可能需要将代码单元格<code>[2]</code>中的<code>PAPER_EXPORT_PATH</code>常量修改为本地机器上的合适导出路径。运行该Notebook需确保系统已安装TeX环境，因为图表通过<code>pdflatex</code>与<code>matplotlib</code>的PGF后端生成。

#### 补充数据
<code>projects</code>目录包含了本论文中提及的所有对应版本的待测项目；<code>run-logs</code>目录则包含了本团队执行实验时生成的全部运行日志。