Metavolve-Labs/supervision-tradeoff

该数据集是The Supervision Tradeoff — Reproducibility Bundle的组成部分，旨在支持后训练、对齐评估、LLM作为评判者协议和校准的研究。它包含一个953提示的OOD确认性语料库（来自MATH-500 L5、HumanEval+、MMLU-Pro和BCB），用于评估；多个学生模型输出（8种训练方案在相同提示下的结果），用于复现验证器分数；种子复制数据（包括完整和交集版本），用于展示训练不稳定性；以及推理NEST v2试点语料库（100条记录），用于失败拓扑训练。数据集还提供了原始评判JSON文件、聚合指标、分析脚本和训练代码，以支持方法复制和进一步研究。主要发现包括对齐税、校准强迫、密度回退和复制危机，强调了在小规模语料库微调中种子脆弱性的重要性。

This dataset is part of The Supervision Tradeoff — Reproducibility Bundle, designed to support research in post-training, alignment evaluation, LLM-as-judge protocols, and calibration. It includes a 953-prompt OOD confirmatory corpus (from MATH-500 L5, HumanEval+, MMLU-Pro, and BCB) for evaluation; multiple student model outputs (8 training arms on the same prompts) to reproduce verifier scores; seed replication data (full and intersection versions) to demonstrate training instability; and a Reasoning-NEST v2 pilot corpus (100 records) for failure-topology training. The dataset also provides raw judgment JSON files, aggregate metrics, analysis scripts, and training code to facilitate methodological replication and further research. Key findings include the alignment tax, calibration compulsion, density walk-back, and the replication crisis, highlighting the importance of seed fragility in small-corpus fine-tuning.