Replication Package: Output Format x Model Identity Interaction Effects in Single-Round Coding Agent Performance
收藏资源简介:
Replication package for the paper "Output Format x Model Identity: Interaction Effects in Single-Round Coding Agent Performance" (arXiv preprint). Contents: 3,976 controlled single-round coding agent runs across 3 models (DeepSeek V4, Doubao 2.0 Pro, Qwen 3.7 Max), 4 projects (tqdm, dotenv, requests, jsoup), 3 output formats (full_file, json_patch, unified_diff). Includes first-N deduplicated core dataset (900 runs, P2 tqdm), full-data CSV with per-run metrics (success, tokens, cost_rmb, error_msg), and experiment prompt templates (Appendix A). Raw agent outputs (~9GB) archived locally; available upon request for journal peer review. Total API cost: Y438.99 (~$61 USD). Breakdown: DeepSeek V4 Y35.86 (99% cache), Doubao 2.0 Pro Y219.18, Qwen 3.7 Max Y183.95. Note: DeepSeek cost reflects experimental repeated-task cache-hit rate; single-shot scenarios expect 5-10x higher costs.



