遇见数据集

BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests

收藏
Zenodo2026-07-31 更新2026-08-01 收录
官方服务:

资源简介:

BulkPR-Bench measures whether a coding agent can govern a batch of interacting candidate pull requests in one repository as a whole: recover the relations that decide safety and the optimal choice, select a safe and maximal subset, and produce a merge order that actually executes. This release contains 18 frozen repository pools with 581 constructed candidate pull requests, the complete ground truth (relation rules, must-reject sets, and the oracle optimum with its proof), the task generator, the language test gate adapters, and the scoring and metric code including RDS, the ranking metric. Building the task tree from it yields 1232 tasks with the frozen hash beb3a6222a71a55b8bd594cde24716cfedd5bee9265d93e9765c81153f46d368. The README and docs/RUN-PROTOCOL.md provide the commands for building the task tree, running the paper experiment settings with Harbor, and scoring the results. Licensing is in three layers: our code is MIT; our original metadata and annotations (relation rules, oracle proofs, projection schema) are CC BY 4.0; the upstream code inside data/pools/*/diffs/ remains under its source repository's license. All 18 upstream repositories are non-copyleft (MIT, Apache-2.0, BSD-3-Clause, Apache-2.0 OR BSD-2-Clause, ISC). See LICENSE, LICENSE-DATA, NOTICE and REUSE.toml in the artifact.

提供机构:
Zenodo
创建时间:
2026-07-31
二维码
社区交流群
二维码
科研交流群
商业服务