SYSUSELab/FeedbackEval

FeedbackEval是一个用于评估大型语言模型在反馈驱动的代码修复能力的数据集。包含394个编码任务，覆盖了多样的编程场景，每个任务包含3,736个错误代码实例，每个实例配以四种不同类型的反馈：文档字符串、上下文、测试反馈、编译器反馈、人类反馈和简单反馈。FeedbackEval is a benchmark dataset for evaluating the ability of large language models in feedback-driven code repair. It consists of 394 coding tasks covering a diverse range of programming scenarios, with each task including 3,736 erroneous code instances, each paired with four distinct types of feedback: docstring, context, test feedback, compiler feedback, human feedback, and simple feedback.