遇见数据集

Replication Package: Output Format x Model Identity Interaction Effects in Single-Round Coding Agent Performance

收藏
Zenodo2026-07-22 更新2026-08-01 收录
官方服务:

资源简介:

Replication package for the paper "Output Format x Model Identity: Interaction Effects in Single-Round Coding Agent Performance" (arXiv preprint). Contents: 3,976 controlled single-round coding agent runs across 3 models (DeepSeek V4, Doubao 2.0 Pro, Qwen 3.7 Max), 4 projects (tqdm, dotenv, requests, jsoup), 3 output formats (full_file, json_patch, unified_diff). Includes first-N deduplicated core dataset (900 runs, P2 tqdm), full-data CSV with per-run metrics (success, tokens, cost_rmb, error_msg), and experiment prompt templates (Appendix A). Raw agent outputs (~9GB) archived locally; available upon request for journal peer review. Total API cost: Y438.99 (~$61 USD). Breakdown: DeepSeek V4 Y35.86 (99% cache), Doubao 2.0 Pro Y219.18, Qwen 3.7 Max Y183.95. Note: DeepSeek cost reflects experimental repeated-task cache-hit rate; single-shot scenarios expect 5-10x higher costs.

提供机构:
Zenodo
创建时间:
2026-07-22
二维码
社区交流群
二维码
科研交流群
商业服务