humaneval-x

HumanEval-X是一个用于评估代码生成模型多语言能力的基准。它包含820个高质量的人工数据样本，涵盖Python、C++、Java、JavaScript和Go五种编程语言，每个样本都带有测试用例。该数据集支持代码生成和翻译等多种任务，并提供标准化数据操作，用户可以指定编程语言子集进行加载。数据集中的字段包括任务ID、提示、声明、标准解决方案、测试用例和示例测试用例。该数据集采用Apache 2.0授权许可。

HumanEval-X is a benchmark developed for evaluating the multilingual capabilities of code generation models. It comprises 820 high-quality manually curated data samples across five programming languages: Python, C++, Java, JavaScript, and Go, with each sample paired with corresponding test cases. This dataset supports multiple tasks including code generation and code translation, and offers standardized data manipulation functionalities, enabling users to load specified subsets of the included programming languages. The dataset includes the following fields: task ID, prompt, declaration, standard solution, test cases, and example test cases. It is licensed under the Apache 2.0 License.

搜集汇总

背景与挑战

背景概述

HumanEval-X是一个用于评估代码生成模型多语言能力的基准数据集，包含820个高质量人工编写的数据样本，覆盖Python、C++、Java、JavaScript和Go五种编程语言，每个样本均带有测试用例，支持代码生成和翻译等任务，采用Apache 2.0授权许可。