Bounded Returns to Orchestration: Reproduction Data & Analysis Code
收藏资源简介:
Reproduction data and analysis code for the paper 'Bounded Returns to Orchestration: A Controlled Comparison of Eleven Deep-Research Architectures on a Shared Tool Layer'. The archive contains the paper PDF and LaTeX source, the canonical results store (a JSON store holding the numbers behind every table and figure in the paper), the per-result analysis scripts, the rebuild_all.sh reproduction entrypoint, the result tables the paper inputs, and the rendered figures. Given the results store, every table and figure regenerates from the scripts, and two reconciliation gates verify the release: every numeric cell of every table the paper inputs traces to the store (388/388), and the 81 load-bearing inline statistics of the abstract, contributions, and results are pinned to their store keys (zero drift). Raw per-query judge outputs and model checkpoints are not included; version 1 of this record carries the full verdict parquets, which are unchanged in v2. See README.md for full contents, the mapping to the paper, and reproduction instructions. Version 2 (July 2026) accompanies the revised paper and adds: length-adjusted headline contrasts; admissible per-judge citation-regression inference (query-clustered and exact wild-cluster bootstrap); scale-free per-judge orchestration effect sizes; capability gaps on the contamination-clean residual; a reversed-split sensitivity for the decoupled best-of-N control; Holm-corrected equivalence counts; per-dimension confidence intervals for the matched-budget probe; an arm-clustered search-robustness fit; a bootstrap CI for the oracle gap compression; a partial-support bracketing sensitivity for the entailment decomposition; regenerated figures; and the prose-reconciliation harness.



