BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests
收藏资源简介:
BulkPR-Bench measures whether a coding agent can govern a batch of interacting candidate pull requests in one repository as a whole: recover the relations that decide safety and the optimal choice, select a safe and maximal subset, and produce a merge order that actually executes. This release contains 18 frozen repository pools with 581 constructed candidate pull requests, the complete ground truth (relation rules, must-reject sets, and the oracle optimum with its proof), the task generator, the language test gate adapters, and the scoring and metric code including RDS, the ranking metric. Building the task tree from it yields 1232 tasks with the frozen hash beb3a6222a71a55b8bd594cde24716cfedd5bee9265d93e9765c81153f46d368. The README and docs/RUN-PROTOCOL.md provide the commands for building the task tree, running the paper experiment settings with Harbor, and scoring the results. Licensing is in three layers: our code is MIT; our original metadata and annotations (relation rules, oracle proofs, projection schema) are CC BY 4.0; the upstream code inside data/pools/*/diffs/ remains under its source repository's license. All 18 upstream repositories are non-copyleft (MIT, Apache-2.0, BSD-3-Clause, Apache-2.0 OR BSD-2-Clause, ISC). See LICENSE, LICENSE-DATA, NOTICE and REUSE.toml in the artifact.



