HumanEval-ET

该数据集名为HumanEval-ET，是HumanEval的增强版，其中引入了额外的隐藏测试用例，以提高代码评估的可靠性。由于其包含了覆盖边缘情况的额外隐藏测试，因此被认为是一个更加可靠的基准。该数据集涵盖了不同规模的问题，大约有100个额外的隐藏测试用例，任务类型为代码生成。

This dataset, named HumanEval-ET, is an enhanced variant of the original HumanEval dataset. It integrates additional hidden test cases to enhance the reliability of code evaluation. As it incorporates supplementary hidden tests that cover edge cases, it is recognized as a more robust benchmark for code evaluation tasks. This dataset encompasses problems of varying scales, includes approximately 100 additional hidden test cases, and is designed for code generation tasks.

搜集汇总

背景与挑战

背景概述

HumanEval-ET是用于评估代码生成的数据集，属于CodeGenEvaluation项目的一部分，旨在通过代码执行学习来提升代码生成的评估效果。