HumanEval-MRI-Cpp
收藏HumanEval-MRI-Cpp 数据集概述
数据集基本信息
- 源数据集: zai-org/humaneval-x
- 下载大小: 148031 bytes
- 数据集大小: 834118 bytes
- 测试集样本数: 473
- 测试集大小: 834118 bytes
数据集特征
- instance_id: 字符串类型，唯一标识实例
- number_spans: 整型，表示移除的跨度数量
- prompt: 字符串类型，问题描述
- declaration: 字符串类型，函数声明
- splits: 字符串序列，分割后的代码片段
- removed_spans: 字符串序列，被移除的代码片段
- canonical_solution: 字符串类型，标准解决方案
- test: 字符串类型，测试用例
数据集用途
- 该数据集是C++翻译版HumanEval的多区域填充版本，通过从标准解决方案中随机移除1到3个跨度生成。
- 用于论文《Constrained Decoding of Diffusion LLMs with Context-Free Grammars》中的评估。
示例使用
python from datasets import load_dataset import json
dataset = load_dataset(eth-sri/HumanEval-MRI-Cpp) for instance in dataset[test]: print(json.dumps(instance, indent=2)) break
示例实例
json { "instance_id": "CPP/0_spans_1", "number_spans": 1, "prompt": "/* Check if in given vector of numbers, are any two numbers closer to each other than given threshold.
has_close_elements({1.0, 2.0, 3.0}, 0.5) false has_close_elements({1.0, 2.8, 3.0, 4.0, 5.0, 2.0}, 0.3) true */ #include<stdio.h> #include<vector> #include<math.h> using namespace std; bool has_close_elements(vector<float> numbers, float threshold){ ", "declaration": "#include<stdio.h> #include<vector> #include<math.h> using namespace std; #include<algorithm> #include<stdlib.h> bool has_close_elements(vector<float> numbers, float threshold){ ", "splits": [ " int i,j;
for (i=0;i<numbers.size();i++)
for (j=i+1;j<numbers.size();j++)
if ",
";
return false;
}
" ], "removed_spans": [ "(abs(numbers[i]-numbers[j])<threshold) return true" ], "canonical_solution": " int i,j;
for (i=0;i<numbers.size();i++)
for (j=i+1;j<numbers.size();j++)
if (abs(numbers[i]-numbers[j])<threshold)
return true;
return false;
}
", "test": "#undef NDEBUG #include<assert.h> int main(){ vector<float> a={1.0, 2.0, 3.9, 4.0, 5.0, 2.2}; assert (has_close_elements(a, 0.3)==true); assert (has_close_elements(a, 0.05) == false);
assert (has_close_elements({1.0, 2.0, 5.9, 4.0, 5.0}, 0.95) == true);
assert (has_close_elements({1.0, 2.0, 5.9, 4.0, 5.0}, 0.8) ==false);
assert (has_close_elements({1.0, 2.0, 3.0, 4.0, 5.0}, 2.0) == true);
assert (has_close_elements({1.1, 2.2, 3.1, 4.1, 5.1}, 1.0) == true);
assert (has_close_elements({1.1, 2.2, 3.1, 4.1, 5.1}, 0.5) == false);
} " }
