LLMSecEval

LLMSecEval数据集由汉堡工业大学软件安全研究所创建，包含150个自然语言提示，用于评估大型语言模型生成的代码安全性。该数据集涵盖了MITRE的Top 25 CWE中的18种常见弱点，每个提示附有安全实现示例，以支持与LLM生成代码的比较评估。数据集的创建过程涉及使用Codex从相关代码片段生成自然语言描述，并经过手动筛选和格式化。LLMSecEval旨在解决自动代码生成中的安全问题，特别是在自然语言驱动的软件部署中。

The LLMSecEval dataset was created by the Institute of Software Security at Hamburg University of Technology, consisting of 150 natural language prompts for evaluating the code safety of outputs generated by Large Language Models (LLMs). This dataset covers 18 common weaknesses from MITRE’s Top 25 CWE list, and each prompt is paired with a secure implementation example to support comparative evaluation against LLM-generated code. The dataset creation process involved using Codex to generate natural language descriptions from relevant code snippets, followed by manual filtering and formatting. LLMSecEval aims to address security issues in automated code generation, especially in natural language-driven software deployment.