c01dsnap/LLM-Sec-Evaluation

---
license: cc-by-nc-sa-4.0
---
# LLM Security Evaluation
This repo contains scripts for evaluating LLM security abilities. We gathered hundreds of questions cover different ascepts of security, such as vulnerablities, pentest, threat intelligence, etc.

All the questions can be viewed at [https://huggingface.co/datasets/c01dsnap/LLM-Sec-Evaluation](https://huggingface.co/datasets/c01dsnap/LLM-Sec-Evaluation).

## Suppoted LLM
* ChatGLM
* Baichuan
* Vicuna ([GGML format](https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GGML))

## Usage
Because of different LLM requires for different running environment, we highly recommended to manage your virtual envs via Miniconda.

1.Install dependencies
```bash
pip install -r requirements.txt

# If you want to use GPU, please install llama-cpp-python with the following command
LLAMA_CUBLAS=1 CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --verbose
```
2.Clone this repo
```bash
git clone https://github.com/Coldwave96/LLM-Sec-Evaluation
cd LLM-Sec-Evaluation
```
3.Run bash scripts
```bash
# You might need to modify the script running interpreter in evaluate.py
bash evaluate.sh
```

## Changelog
- 2023.7.13 - Add support for ChatGLM & Baichuan
- 2023.7.17 - Add support for Vicuna

原始信息汇总

数据集概述

数据集名称

LLM Security Evaluation

数据集内容

本数据集包含数百个问题，涵盖安全领域的多个方面，如漏洞、渗透测试、威胁情报等。

数据集访问

数据集中的所有问题可在以下链接查看：https://huggingface.co/datasets/c01dsnap/LLM-Sec-Evaluation

支持的LLM模型

• ChatGLM
• Baichuan
• Vicuna (GGML格式)

使用方法

建议使用Miniconda管理虚拟环境，并按照以下步骤操作：

1. 安装依赖：pip install -r requirements.txt
2. 克隆仓库：git clone https://github.com/Coldwave96/LLM-Sec-Evaluation
3. 运行脚本：bash evaluate.sh

更新日志

• 2023.7.13 - 新增对ChatGLM和Baichuan的支持
• 2023.7.17 - 新增对Vicuna的支持

搜集汇总

背景与挑战

背景概述

该数据集名为'LLM-Sec-Evaluation'，是一个用于评估大型语言模型安全能力的资源，包含数百个覆盖漏洞、渗透测试和威胁情报等安全领域的问题。它支持ChatGLM、Baichuan和Vicuna等特定模型，并提供详细的使用脚本和更新日志，以帮助用户进行安全测试和分析。