devopseval-exam

<!--- 以上YAML section提供属性/tags描述--->

<!--- 以下为markdown格式的dataset描述--->

DevOps-Eval is a comprehensive chinese evaluation suite specifically designed for foundation models in the DevOps field. It consists of 4850 multi-choice questions spanning 53 diverse categories. Please visit our website and [GitHub](https://github.com/codefuse-ai/codefuse-devops-eval) for more details.

Each category consists of two splits: dev, and test. The dev set per subject consists of five exemplars with explanations for few-shot evaluation. And the test set is for model evaluation. Labels on the test split are released, users can evaluate their results and automatically obtain test accuracy. [How to evaluate](https://github.com/codefuse-ai/codefuse-devops-eval#-how-to-evaluate)?

#### Notes
More details on loading and using the data are at our github page [github](https://github.com/codefuse-ai/codefuse-devops-eval) page.

DevOps-Eval 是一款专为 DevOps 领域基础模型（Foundation Model）打造的综合性中文评测套件。它包含 4850 道多项选择题，涵盖 53 个多样化的任务类别。如需了解更多详情，请访问我们的官方网站及 [GitHub](https://github.com/codefuse-ai/codefuse-devops-eval) 页面。

每个任务类别均包含两个子集：开发集（dev）与测试集（test）。每个类别的开发集包含 5 道附带解析的示例题，用于少样本（Few-shot）评估；测试集则用于模型性能评测。测试集的标签已公开，用户可自行提交评测结果并自动获取测试准确率。[如何进行评测](https://github.com/codefuse-ai/codefuse-devops-eval#-how-to-evaluate)？

#### 注意事项
关于数据集的加载与使用的更多细节，请参阅我们的 GitHub 页面 [GitHub](https://github.com/codefuse-ai/codefuse-devops-eval)。