Beyond One-to-One: A Multi-Agent Framework for Unit Test Co-Evolution

Replication Package of Beyond One-to-One: A Multi-Agent Framework for Unit Test Co-Evolution This is the replication package for TestPilot, a cascaded LLM-based approach for repairing broken unit tests after production code changes. TestPilot routes each case through three stages, escalating only on failure: MULTI (whole-class generation) → REPAIR (in-place fix of MULTI's failures) → TU (per-method regeneration). All methods are evaluated by a single unified `post_eval` (clean-rebuild) for a fair comparison. What's here - The **TestPilot method** (`src/co_evo/testpilot_runner/testpilot.py` + the `multi_engine/` MULTI engine).- The **baselines** it is compared against: TestUpdater, SynTeR, Agentic, NaiveLLM.- The **unified evaluation** (`baseline_post_eval/`) — one clean-rebuild ruler for every method.- The **dataset** (`data/`) and the **dataset-mining scripts** (`data_collection/`).- TestPilot's **generated outputs** for reference (`data/baseline_outputs/`). Quantitative results and the per-baseline analysis are reported in the paper. Contents ```.├── src/co_evo/│ ├── testpilot_runner/testpilot.py # ★ TestPilot main entry (MULTI→REPAIR→TU)│ ├── baseline_post_eval/ # ★ unified evaluation (the single ruler)│ ├── {testupdater,synter,agentic}_original_runner/ # baselines│ └── ...├── multi_engine/ # ★ the MULTI generation engine├── data_collection/ # ★ dataset mining scripts (how the dataset was built)├── data/│ ├── baselines/coevo_nm_249.json # 249 methods (flat; keyed by raw_id)│ ├── strict_nowtest/*.json # 28 repos grouped (TestPilot input)│ └── baseline_outputs/qwen/testpilot.jsonl # TestPilot's generated outputs (reference)├── reproduce.sh # one-command reproduction├── warm_m2.sh # pre-warm shared Maven .m2 (run once)├── REPRODUCE.md # ★ detailed reproduction guide└── requirements.txt``` Quick start ```bashpip install -r requirements.txt# See REPRODUCE.md for full setup (JDK/Maven/gradle, benchmark repos, .m2 warm-up, API keys).bash reproduce.sh testpilot qwen3.5-plus # run TestPilot under Qwen``` The dataset is keyed by **`raw_id`** (a stable content hash); `test_id` is a clean 0–248 index.See **REPRODUCE.md** for runtime dependencies and the `.m2` dependency-freezing step (some benchmarkSNAPSHOT dependencies have been removed upstream). Notes - All baselines are judged by the same `baseline_post_eval.core` (clean-rebuild) so results are directly comparable. Cost is measured in LLM tokens/case (wall-clock is dominated by API latency).- Models: qwen3.5-plus, deepseek-v4-pro, gpt-5.2, claude-opus-4-6.- Running the full pipeline requires the documented runtime environment (JDKs / Maven / gradle / jdtls / benchmark repos / API keys — see **REPRODUCE.md**); the pipeline has been validated in the authors' environment. The per-machine tool-path configuration in `multi_engine/utils/configs.py` is driven by environment variables; a fully turnkey, environment-independent runner is planned for the camera-ready revision.