TreeBench-v3: A De-leaked Hierarchical QA Benchmark and Audit Artifacts
收藏资源简介:
TreeBench-v1.1 is the corrected and de-leaked release associated with the revised TreeBench-Audit manuscript. This version supersedes TreeBench-v1.0 for evaluation. An audit of v1.0 found evidence leakage in the generated questions and an ID-normalization artifact affecting required-node recall. v1.1 preserves the original corpus, tree annotations, and failure taxonomy, but replaces the question text with a leakage-hardened 363-question subset that has no residual ≥8-word verbatim question–evidence overlap under the released detector. TreeBench-v1.0 is retained only for reproducibility of the audit and paired-control experiments. New evaluations should use TreeBench-v1.1. Main changes in v1.1:- Adds TreeBench-v1.1, a de-leaked 363-question subset.- Marks TreeBench-v1.0 as deprecated for evaluation.- Fixes normalized node-ID matching used for required-node recall.- Adds leakage detector outputs and audit artifacts.- Adds paired leaky-vs-de-leaked evaluation results.- Adds causal ablation results for confounder removal and path augmentation.- Updates documentation, dataset card, and reproducibility notes. Related code, detector, and audit trail:https://github.com/whitepaper27/TreeBench



