Req2Test: An Approach for the Automatic Generation of Unit Tests from User Stories Using LLMs

This Zenodo repository contains supplementary artifacts for the article: Req2Test: An Approach for the Automatic Generation of Unit Tests from User Stories Using LLMs. It provides datasets, generated tests, and experimental materials to support reproducibility and evaluation of the proposed approach. Repository Contents Evaluation 1. Phase1 - Controlled Experiment 1.1 Specifications – User Stories and DSL Specifications of four projects in User Story format and their corresponding Domain-Specific Language (DSL) representations. 1.2. Generated Unit Tests Unit tests generated by the Req2Test Tool for the four projects, covering various use cases and system functionalities. 1.3. Generated Support Artifacts Artifacts produced during the test generation pipeline, including: - Test scripts - Json data test - Test cases 1.4. Generated Prompts Prompts created by Req2Test Tool for: - Test data generation - Automated generation of unit test classes These prompts serve as a reproducibility baseline, since executing them sequentially leads to the generation of the unit test classes. They are produced by Req2Test Tool, which uses the DSL and test templates to create the prompts that guide the automated creation of the tests. 1.5. Projects of Controlled Experiment Detailed materials for the controlled experiment, including: - Experiment instructions (with links to the forms used) - Injected bugs - Participant response spreadsheets - Projects used in the experiment: - buggy-without-tests - buggy-with-tests - correct-with-tests - correct-without-tests The original materials are in Portuguese; English translations are also provided. Projects Requirements - Java 17 or higher - Maven (for building and running generated tests) 2. Phase2- Comparison - EvoSuite x Req2Test 2.1 reports-instruction-coverage-and-mutation-score Detailed results for five runs across the four projects using Req2Test and EvoSuite, including Instruction Coverage and Mutation Score. 2.2 Projects - Tests generated using EvoSuite All four projects were executed with five runs each using EvoSuite. The generated tests are available in the directories evosuite-tests-run1 to evosuite-tests-run5, under src/test/java and src/test/resources. Projects Requirements - Java 8 - Maven (for building and running generated tests) 2.3 Projects - Tests generated using Req2Test Generated Artefacts – All unit tests and prompts produced for the four projects, with five runs per project. Projects - All four projects were executed with five runs each using Req2Test. The generated tests are available under src/test/java and src/test/resources. Projects Requirements - Java 17 - Maven (for building and running generated tests) 3. Phase3 - QuixBugs 3.1 Req2Test-Generated All unit tests and prompts produced for the four projects, with five runs per project. 3.1 Correct A project with 24 correct algorithms adapted to the Req2Test architecture, including tests for all 24 algorithms generated with Req2Test, each executed five times. DSL DSL-Projects-and-QuixBugs DSL-Projects- This document presents the project specifications from the dataset-api repository, written in the Req2Test DSL. It includes four REST API projects, detailing their business rules, validation constraints, and API responses. DSL-Quixbugs - This document presents the 24 QuixBugs algorithms written in the Req2Test DSL, describing each algorithm’s inputs, constraints, and expected behavior. Related Repository The repository and tool used to generate these artifacts are available at: Req2Test Tool - https://github.com/specgentest/req2test-tool API dataset of the projects used in the experiment - https://github.com/specgentest/dataset-api