xbeark/JailbreakHub

---
language:
- en
dataset_info:
features:
- name: prompt
dtype: string
- name: platform
dtype: string
- name: source
dtype: string
- name: jailbreak
dtype: bool
splits:
- name: train
num_bytes: 27480306
num_examples: 15140
download_size: 14209695
dataset_size: 27480306
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
license: mit
---

# In-The-Wild Jailbreak Prompts on LLMs

**Paper**: [``Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://arxiv.org/pdf/2308.03825)

**Data**: [Dataset](https://github.com/verazuo/jailbreak_llms/tree/main/data/prompts)

## Data

## Prompts

Overall, authors collect 15,140 prompts from four platforms (Reddit, Discord, websites, and open-source datasets) during Dec 2022 to Dec 2023. Among these prompts, they identify 1,405 jailbreak prompts. To the best of our knowledge, this dataset serves as the largest collection of in-the-wild jailbreak prompts.

The data are provided [here](https://github.com/verazuo/jailbreak_llms/tree/main/data).

> Statistics of our data source. (Adv) UA refers to (adversarial) user accounts.

| Platform | Source | # Posts | # UA | # Adv UA | # Prompts | # Jailbreaks | Prompt Time Range |
| --------- | -------------------------- | ----------- | --------- | -------- | ---------- | ------------ | ------------------- |
| Reddit | r/ChatGPT | 163549 | 147 | 147 | 176 | 176 | 2023.02-2023.11 |
| Reddit | r/ChatGPTPromptGenius | 3536 | 305 | 21 | 654 | 24 | 2022.12-2023.11 |
| Reddit | r/ChatGPTJailbreak | 1602 | 183 | 183 | 225 | 225 | 2023.02-2023.11 |
| Discord | ChatGPT | 609 | 259 | 106 | 544 | 214 | 2023.02-2023.12 |
| Discord | ChatGPT Prompt Engineering | 321 | 96 | 37 | 278 | 67 | 2022.12-2023.12 |
| Discord | Spreadsheet Warriors | 71 | 3 | 3 | 61 | 61 | 2022.12-2023.09 |
| Discord | AI Prompt Sharing | 25 | 19 | 13 | 24 | 17 | 2023.03-2023.04 |
| Discord | LLM Promptwriting | 184 | 64 | 41 | 167 | 78 | 2023.03-2023.12 |
| Discord | BreakGPT | 36 | 10 | 10 | 32 | 32 | 2023.04-2023.09 |
| Website | AIPRM | - | 2777 | 23 | 3930 | 25 | 2023.01-2023.06 |
| Website | FlowGPT | - | 3505 | 254 | 8754 | 405 | 2022.12-2023.12 |
| Website | JailbreakChat | - | - | - | 79 | 79 | 2023.02-2023.05 |
| Dataset | AwesomeChatGPTPrompts | - | - | - | 166 | 2 | - |
| Dataset | OCR-Prompts | - | - | - | 50 | 0 | - |
| **Total** | | **169,933** | **7,308** | **803** | **15,140** | **1,405** | **2022.12-2023.12** |

## License
`jailbreak_llms` is licensed under the terms of the MIT license. See LICENSE for more details.

## Citation
If you find this useful in your research, please consider citing:

```
@inproceedings{SCBSZ24,
author = {Xinyue Shen and Zeyuan Chen and Michael Backes and Yun Shen and Yang Zhang},
title = {{``Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models}},
booktitle = {{ACM SIGSAC Conference on Computer and Communications Security (CCS)}},
publisher = {ACM},
year = {2024}
}
```

语言：
- 英语
数据集信息：
特征字段：
- 字段名：prompt（提示），数据类型：字符串
- 字段名：platform（平台），数据类型：字符串
- 字段名：source（来源），数据类型：字符串
- 字段名：jailbreak（越狱），数据类型：布尔值
数据集划分：
- 划分集名称：训练集（train），字节数：27480306，样本数量：15140
下载大小：14209695
数据集总大小：27480306
配置项：
- 配置名称：默认（default），数据文件：
- 划分集：训练集，路径：data/train-*
许可证：MIT许可证

# 面向大语言模型（LLM）的野外越狱提示
**论文**：[《“为所欲为”：大语言模型野外越狱提示的特征分析与评估》](https://arxiv.org/pdf/2308.03825)
**数据集**：[数据集链接](https://github.com/verazuo/jailbreak_llms/tree/main/data/prompts)

## 数据集概况
## 提示样本
总体而言，研究团队于2022年12月至2023年12月期间，从四大平台（Reddit、Discord、公开网站及开源数据集）共收集15140条提示样本，其中识别出1405条越狱提示。据我们所知，本数据集是目前规模最大的野外越狱提示集。

完整数据集可通过[此链接](https://github.com/verazuo/jailbreak_llms/tree/main/data)获取。

> 本数据集来源统计说明。（Adv）UA 指（对抗性）用户账户。

| 平台 | 来源 | 帖子数 | 用户账户数（UA） | 对抗性用户账户数（Adv UA） | 提示样本数 | 越狱提示数 | 时间范围 |
| ---------- | ------------------------ | ---------- | -------------- | ------------------------ | ---------- | ---------- | -------------------- |
| Reddit | r/ChatGPT | 163549 | 147 | 147 | 176 | 176 | 2023.02-2023.11 |
| Reddit | r/ChatGPTPromptGenius | 3536 | 305 | 21 | 654 | 24 | 2022.12-2023.11 |
| Reddit | r/ChatGPTJailbreak | 1602 | 183 | 183 | 225 | 225 | 2023.02-2023.11 |
| Discord | ChatGPT | 609 | 259 | 106 | 544 | 214 | 2023.02-2023.12 |
| Discord | ChatGPT Prompt Engineering | 321 | 96 | 37 | 278 | 67 | 2022.12-2023.12 |
| Discord | Spreadsheet Warriors | 71 | 3 | 3 | 61 | 61 | 2022.12-2023.09 |
| Discord | AI Prompt Sharing | 25 | 19 | 13 | 24 | 17 | 2023.03-2023.04 |
| Discord | LLM Promptwriting | 184 | 64 | 41 | 167 | 78 | 2023.03-2023.12 |
| Discord | BreakGPT | 36 | 10 | 10 | 32 | 32 | 2023.04-2023.09 |
| 网站 | AIPRM | - | 2777 | 23 | 3930 | 25 | 2023.01-2023.06 |
| 网站 | FlowGPT | - | 3505 | 254 | 8754 | 405 | 2022.12-2023.12 |
| 网站 | JailbreakChat | - | - | - | 79 | 79 | 2023.02-2023.05 |
| 开源数据集 | AwesomeChatGPTPrompts | - | - | - | 166 | 2 | - |
| 开源数据集 | OCR-Prompts | - | - | - | 50 | 0 | - |
| **总计** | | **169933** | **7308** | **803** | **15140** | **1405** | **2022.12-2023.12** |

## 许可证
`jailbreak_llms` 采用MIT许可证进行授权，详细条款请参见LICENSE文件。

## 引用方式
若本数据集对你的研究有所帮助，请引用如下文献：
bibtex
@inproceedings{SCBSZ24,
author = {Xinyue Shen and Zeyuan Chen and Michael Backes and Yun Shen and Yang Zhang},
title = {{"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models}},
booktitle = {{ACM SIGSAC Conference on Computer and Communications Security (CCS)}},
publisher = {ACM},
year = {2024}
}