ENAMEL

ENAMEL数据集是由伊利诺伊大学厄巴纳-香槟分校和Qualcomm AI Research合作创建，专注于评估大型语言模型（LLMs）生成的代码的效率。该数据集包含142个问题，这些问题是从164个问题中精心挑选出来的，旨在排除时间复杂度为Θ(1)的简单问题。数据集的创建过程涉及专家设计的高效算法和实现作为参考解决方案，以及强大的测试用例生成器来确保严格的评估。ENAMEL的应用领域主要集中在提高系统吞吐量、改善算法延迟和减少能源消耗，旨在解决当前LLMs在代码生成效率方面的不足。

ENAMEL is a dataset co-created by the University of Illinois Urbana-Champaign and Qualcomm AI Research, focused on evaluating the efficiency of code generated by Large Language Models (LLMs). It includes 142 problems carefully selected from a total of 164 questions, with the intent to exclude simple problems with a time complexity of Θ(1). The development of the ENAMEL dataset incorporates expert-designed efficient algorithms and implementations as reference solutions, as well as a robust test case generator to ensure rigorous evaluation. The primary application domains of ENAMEL center around improving system throughput, optimizing algorithm latency, and reducing energy consumption, with the aim of addressing the current deficiencies of LLMs in code generation efficiency.