RealHumanEval

该数据集是通过一项用户研究收集的，研究中参与者与大型语言模型（LLMs）互动，以测量他们在编程任务中的生产效率，这些任务主要通过自动补全和聊天支持来完成。具体数据显示，参与者在不同LLM支持模型的帮助下，平均在334秒内完成了编程任务。该研究的规模涉及213名参与者和771个编程任务，其任务是评估编程任务的生产效率。

This dataset was collected via a user study where participants interacted with Large Language Models (LLMs) to measure their productivity in programming tasks, which were primarily supported by code autocompletion and chat-based assistance. The study's data demonstrates that, with the assistance of various LLM-powered models, participants completed their programming tasks in an average of 334 seconds. This research involved 213 participants and 771 programming tasks, with the aim of evaluating productivity in programming tasks.