ai-coscientist/researcher-ablation-bench

ResearcherAblationBench是一个用于评估语言模型在实证AI研究中进行消融计划的能力的基准测试套件的一部分，名为AblationBench。该数据集包含83篇研究论文和230个由人类注释的消融计划，旨在帮助作者根据论文的方法部分生成消融计划。数据集提供了基于自动语言模型的评估器，用于评估生成的计划与注释之间的匹配程度。

ResearcherAblationBench is part of AblationBench, a benchmark suite for evaluating language models on ablation planning in empirical AI research. It focuses on assisting authors by testing a models ability to generate ablation plans based solely on a papers method section. The dataset consists of 83 research papers with 230 human-annotated ablation plans, aiming to assist authors in proposing ablation plans based on their papers method section.