Req2Test: An LLM-Based Approach for Automatic Generation of Integration Tests from DSL Specifications for RESTful APIs

This Zenodo repository contains supplementary artifacts for the paper: Req2Test: An LLM-Based Approach for Automatic Generation of Integration Tests for RESTful APIs from User Story DSL. It provides datasets, generated tests, and experimental materials to support reproducibility and evaluation of the proposed approach. Repository Contents Phase 1 – Controlled Experiment Presents all materials used in the experiment: instructions, injected bugs (including detailed descriptions), participant responses (spreadsheets), profile (spreadsheets), and project versions (correct-without-tests, correct-with-tests, buggy-without-tests, buggy-with-tests). Phase 2 – Comparison (Req2Test x EvoMaster) Presents all materials used in the comparison: data collected in spreadsheets, Req2Test raw results for ten runs across projects P1–P4, project versions (P1–P4) executed with ten Req2Test runs, EvoMaster raw results for ten runs across projects P1–P4, and project versions (P1–P4) executed with ten EvoMaster runs. Phase 3 – QuixBugs (External Validity) Presents all materials used in the external validity analysis with the 23 QuixBugs algorithms: the 23 algorithms written in the Req2Test DSL, data collected in spreadsheets, the raw generated tests for all 23 algorithms across ten runs, the projects with Req2Test-generated tests for ten runs (both correct and buggy versions), and the projects with the original QuixBugs tests (baseline), also in correct and buggy versions. Phase 4 – Assess Hallucination Presents all materials used to assess hallucination using P1–P4 and the 23 QuixBugs algorithms: data collected in spreadsheets, the P1–P4 projects with Req2Test-generated tests for ten runs, and the QuixBugs project with Req2Test-generated tests for ten runs. Phase 5 – Ablation Study Presents all materials used in the ablation study, including Variant B prompts and source code, the results obtained for Variant B and Req2Test, as well as token consumption and runtime. Example in Paper Presents all prompts generated by Req2Test for the example described in Sections 4 and 5 of the paper, along with the generated test cases and the database dataset. Specifications - User Stories and DSL Presents all specifications used in the experiment, including all user stories for P1–P4, the user stories expressed in the Req2Test DSL, and the 23 QuixBugs algorithms encoded in the Req2Test DSL. Data Analysis Scripts All Python scripts used to analyze the five phases of the experiment. Examples with Generated Prompts Presents one complete execution with all artifacts generated for P1–P4: generated prompts, generated support artifacts, and generated integration tests. Related Repository The repository and tool used to generate these artifacts are available at: Req2Test Tool - https://github.com/specgentest/req2test-tool API dataset of the projects used in the experiment - https://github.com/specgentest/dataset-api