Shuibai12138/crb-datasets

代码修订基准（CRB）是一个受控且可执行的基准，旨在评估扩散语言模型中的纠正行为。该数据集包含带有受控令牌级错误的错误代码样本，能够系统地评估掩码扩散语言模型通过自我修订机制定位和纠正错误的能力。数据集提供了受控的令牌级损坏（操作符、标识符和字面量替换）、错误位置注释（相对于错误代码体的精确令牌级错误位置）、多种模型变体（使用不同分词器生成的样本）以及基于HumanEval、HumanEval+、MBPP和MBPP+数据集的可执行基准。

The Code Revision Benchmark (CRB) is a controlled and executable benchmark designed to evaluate corrective behavior in diffusion language models. This dataset contains buggy code samples with controlled token-level errors, enabling systematic evaluation of how well masked diffusion language models can localize and correct errors through self-revision mechanisms. It provides controlled token-level corruption (operator, identifier, and literal substitutions), error position annotations (precise token-level error locations relative to buggy body), multiple model variants (samples generated using different tokenizers), and executable benchmarks based on HumanEval, HumanEval+, MBPP, and MBPP+ datasets.