RAITG: Requirement-Aware Intelligent Test Generator — Dataset, Pipeline, and Reproducibility Bundle

Reproducibility bundle for RAITG, a 4-stage LLM-based pipeline that generates test cases from natural-language requirements. v2.0.0 (2026-06-17) — What's new ======================================== - 4th Subject Under Test added: logistics-app (last-mile delivery service, 192 lines of Python) - 50 new natural-language requirements covering shipment lifecycle, driver assignment with capacity + hazmat constraints, delivery confirmation, and exception handling - Total dataset: 362 requirements across 4 domains (commercial web 112, financial services 98, healthcare 102, logistics 50) - Full pipeline re-run: 1,086 LLM calls (362 requirements × 3 conditions: unverified, ablation, full RAITG) - Executable mutation testing now covers 4 reference applications: 273 mutants total, 225 killed = 82.42% aggregate kill rate (banking-api 86.84%, hr-app 85.25%, logistics-app 81.01%, fhir-lite 75.44%) - Statistics regenerated for n=362: verification full vs ablation +22.1 percentage points, Cohen's d = 0.65, Wilcoxon p < 0.0001 paired observations - Per-domain breakdown for logistics: 100% coverage, 12.0% symbolic mutation indicator, 98.0% verification pass (highest of all four domains) - Companion preprint v2 included (paperE_preprint_v2.pdf, 11 pages) - Pipeline scripts patched to accept logistics domain; baseline-test bug in hr-app password validator fixed Contents ======================================== - datasets/ — All 362 requirements as JSON (split per domain + combined) - repo/ — 4 SUTs source code (hr-app 142, banking-api 126, fhir-lite 147, logistics-app 192 LOC) - scripts/ — Full RAITG pipeline (run_experiment, prompts, verify, mutation, stats_analysis, generate_all_figures, baseline_tests, llm_adapter) - tables/ — 10 regenerated CSVs (aggregate, per-domain, per-app mutation, per-operator, stats with n=362, bootstrap CIs, paired significance, multi-model) - results/ — Mutation testing JSON + sample run logs per condition - figures/ — All 7 paper figures - paperE_preprint_v2.pdf — Companion paper preprint - README.md — This file (rendered) - CITATION.cff — Machine-readable citation metadata - requirements.txt — Python dependencies To reproduce ======================================== pip install -r requirements.txt export ANTHROPIC_API_KEY="sk-ant-..." python scripts/run_experiment.py --full python scripts/mutation_exec_v2.py python scripts/stats_analysis.py python scripts/generate_all_figures.py Expected runtime: 6-12 hours on commodity hardware. API cost: ~$3-8 (Anthropic Claude Sonnet 4.6). License ======================================== Data and text: CC-BY-4.0 Code: MIT Companion paper ======================================== Paper E (Springer Automated Software Engineering submission, 17 Jun 2026): "LLM-Based Test Case Generation from Natural-Language Requirements: A Verified Multi-Domain Empirical Study with Symbolic Mutation Indicators" Author: Vijay Prasad Javvadi (Independent Researcher, Plainsboro, NJ, USA) ORCID: 0009-0004-1192-6906 GitHub: https://github.com/javvadivijayprasad/TestCaseGen_Reserach AI disclosure ======================================== Anthropic Claude (Sonnet 4.6 + Haiku 4.5 multi-model comparison) is the LLM under evaluation in this work. The same LLM was also used for prose-drafting assistance during manuscript preparation. All empirical claims, dataset extraction, pipeline runs, statistical analyses, and mutation-testing computations were generated by the author's own scripts (included in this bundle) and verified against on-disk artifacts. The AI tool was not used to generate data, results, or analyses. Changelog ======================================== v2.0.0 (2026-06-17) — 4th SUT (logistics-app), 362 reqs, 1086 runs, 273 mutants, n=362 stats v1.0.0 (2026-05-19) — Initial public release, 312 reqs across 3 domains