Study package for "LLM-as-a-Judge Under Version Change: Recalibrate or Re-judge?"
收藏资源简介:
The study package of the paper "LLM-as-a-Judge Under Version Change: Recalibrate or Re-judge?" (Ogous Chan Ali, 2026; preprint at https://ogous.dev/publications/llm-judge-under-version-change, arXiv to follow). The study was pre-registered on OSF (https://osf.io/y6qm8/). Three versions of one judge, Claude Sonnet 4.6, 5 and 5.5, rated the coherence of the 1,600 SummEval summaries (16 systems × 100 articles), each item six times. It holds the registration text, the collection harness, every API attempt and every judge score, both analysis implementations and their outputs, the simulation scripts and records, the post hoc analyses, the script that fills the paper's tables, the converted inputs and manifests that anchorset 0.1.0 (doi:10.5281/zenodo.23358090) recomputes, and the search record. It holds no article or summary text, no request or response text and no copy of SummEval's expert annotations; README.md says how to fetch them. RIGHTS.md gives the licence of each component. SHA256SUMS.txt lists every file.



