CQaaa/iac-eval

IaC-Eval数据集（v1.1版）是第一个针对云基础设施即代码（IaC）领域的人工策划且具有挑战性的数据集，旨在更严格地评估大型语言模型在IaC代码生成方面的能力。该数据集包含458个问题，覆盖了从简单到困难的不同难度级别，并针对多种云服务（目前主要是AWS）。数据集的维护团队已经修复了难度级别4、5和6中的错误，确保这些程序可以无错误或警告地部署到AWS。此外，数据集还提供了AWS Lambda、AWS Elastic Beanstalk、AWS CloudFront等服务的补充程序，进一步丰富了内容。

IaC-Eval dataset (v1.1) is the first human-curated and challenging Cloud Infrastructure-as-Code (IaC) dataset tailored to more rigorously benchmark large language models IaC code generation capabilities. This dataset contains 458 questions ranging from simple to difficult across various cloud services (targeting AWS for now). The maintenance team has resolved bugs in cases with difficulty levels 4, 5, and 6, ensuring the programs can be deployed to AWS without any errors or warnings. Additionally, supplementary programs are now provided for services such as AWS Lambda, AWS Elastic Beanstalk, AWS CloudFront, and others, further enriching the dataset.