遇见数据集

Replication Package: Can Coding Agents Produce Merge-Ready Feature Pull Requests? An Industrial Evaluation of 62 Real Tasks

收藏
Zenodo2026-04-30 更新2026-05-26 收录
官方服务:

资源简介:

Replication package for the ASE 2026 Industry Showcase paper. Contains 62 feature-implementation task definitions (prompts, ground-truth diffs, file classifications), a task registry with commit SHAs and PR URLs, the scoring rubric, evaluation scores for 496 agent runs across four commercial coding agents (Claude Code, Codex CLI, Cursor, OpenCode), and evaluation/analysis scripts.

适用于ASE 2026行业展示论文的复现包,包含62项功能实现任务的定义(含提示词、基准真值代码差异(ground-truth diffs)、文件分类)、带有提交哈希值(commit SHAs)与拉取请求链接(PR URLs)的任务注册表、评分细则,以及覆盖Claude Code、Codex CLI、Cursor、OpenCode四款商业化代码智能体的496次智能体运行评估得分,同时附带评估与分析脚本。

提供机构:
Zenodo
创建时间:
2026-04-30
二维码
社区交流群
二维码
科研交流群
商业服务