Ngixdev/JailBreakV-28k

--- license: mit task_categories: - text-generation - question-answering pretty_name: JailBreakV_28K size_categories: - 10K<n<100K configs: - config_name: JailBreakV_28K data_files: - split: mini_JailBreakV_28K path: JailBreakV_28K/mini_JailBreakV_28K.csv - split: JailBreakV_28K path: JailBreakV_28K/JailBreakV_28K.csv - config_name: RedTeam_2K data_files: - split: RedTeam_2K path: JailBreakV_28K/RedTeam_2K.csv --- # ⛓‍💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks [**🌐 GitHub**](https://github.com/EddyLuo1232/JailBreakV_28K) | [**🛎 Project Page**](https://eddyluo1232.github.io/JailBreakV28K/) ｜ [**👉 Download full datasets**](https://forms.gle/e6boFoM9mNfPHb9H9) ## If you like our project, please give us a star ⭐ on Hugging Face for the latest update. ## 📰 News | Date | Event | |------------|----------| | **2024/07/09** | 🎉 Our paper is accepted by COLM 2024.| | **2024/06/22** | 🛠️ We have updated our version to V0.2, which supports users to customize their attack models and evaluate models.| | **2024/04/04** | 🎁 We have posted our paper on Arxiv.| | **2024/04/03** | 🎉 We have released our evaluation and inference samples.| | **2024/03/30** | 🔥 We have released our dataset.| ## 📥 Using our dataset via huggingface Dataset ```python from datasets import load_dataset mini_JailBreakV_28K = load_dataset("JailbreakV-28K/JailBreakV-28k", 'JailBreakV_28K')["mini_JailBreakV_28K"] JailBreakV_28K = load_dataset("JailbreakV-28K/JailBreakV-28k", 'JailBreakV_28K')["JailBreakV_28K"] RedTeam_2K = load_dataset("JailbreakV-28K/JailBreakV-28k", 'RedTeam_2K')["RedTeam_2K"] ``` ## 👻 Inference and Evaluation ### Create environment ```python conda create -n jbv python=3.9 conda activate jbv pip install -r requirements.txt ``` ### Conduct jailbreak attack on MLLMs ```python # we default use Bunny-v1_0, you can change the default attack model to your customized attack models by editing the annotated codes. # You can follow the Bunny script in <attack_models> to add other attack models. python attack.py --root JailBreakV_28K ``` ### Conduct evaluation ```python # we default use LlamaGuard, you can change the default evaluate model to your customized evaluate models by editing the annotated codes. # You can follow the LlamaGuard script in <evaluate_models> to add other evaluate models. python eval.py --data_path ./results/JailBreakV_28k/<your customized attack model>/JailBreakV_28K.csv ``` ## 😃 Dataset Details JailBreakV_28K and mini_JailBreakV_28K datasets will comprise the following columns: - **id**: Unique identifier for all samples. - **jailbreak_query**: Jailbreak_query obtained by different jailbreak attacks. - **redteam_query**: Harmful query from RedTeam_2K. - **format**: Jailbreak attack method including template, persuade, logic, figstep, query-relevant. - **policy**: The safety policy that redteam_query against. - **image_path**: The file path of the image. - **from**: The source of data. - **selected_mini**: "True" if the data in mini_JailBreakV_28K dataset, otherwise "False". - **transfer_from_llm**: "True" if the jailbreak_query is transferred from LLM jailbreak attacks, otherwise "False". RedTeam_2K will comprise the following columns: - **id**: Unique identifier for all samples. - **question**: Harmful query. - **policy**: the safety policy that redteam_query against. - **from**: The source of data. ## 🚀 Data Composition <!-- Provide a quick summary of the dataset. --> **RedTeam-2K**: RedTeam-2K dataset, a meticulously curated collection of 2, 000 harmful queries aimed at identifying alignment vulnerabilities within LLMs and MLLMs. This dataset spans across 16 safety policies and incorporates queries from 8 distinct sources. <img src="distribution_used.png" width="800"/> **JailBreakV-28K**: JailBreakV-28K contains 28, 000 jailbreak text-image pairs, which include 20, 000 text-based LLM transfer jailbreak attacks and 8, 000 image-based MLLM jailbreak attacks. This dataset covers 16 safety policies and 5 diverse jailbreak methods. ## 🛠️ Dataset Overview The RedTeam-2K dataset, is a meticulously curated collection of 2, 000 harmful queries aimed at identifying alignment vulnerabilities within LLMs and MLLMs. This dataset spans 16 safety policies and incorporates queries from 8 distinct sources, including GPT Rewrite, Handcraft, GPT Generate, LLM Jailbreak Study, AdvBench, BeaverTails, Question Set, and hh-rlhf of Anthropic. Building upon the harmful query dataset provided by RedTeam-2K, JailBreakV-28K is designed as a comprehensive and diversified benchmark for evaluating the transferability of jailbreak attacks from LLMs to MLLMs, as well as assessing the alignment robustness of MLLMs against such attacks. Specifically, JailBreakV-28K contains 28, 000 jailbreak text-image pairs, which include 20, 000 text-based LLM transfer jailbreak attacks and 8, 000 image-based MLLM jailbreak attacks. This dataset covers 16 safety policies and 5 diverse jailbreak methods. The jailbreak methods are formed by 3 types of LLM transfer attacks that include Logic (Cognitive Overload), Persuade (Persuasive Adversarial Prompts), and Template (including both of Greedy Coordinate Gradient and handcrafted strategies), and 2 types of MLLM attacks including FigStep and Query-relevant attack. The JailBreakV-28K offers a broad spectrum of attack methodologies and integrates various image types like Nature, Random Noise, Typography, Stable Diffusion (SD), Blank, and SD+Typography Images. We believe JailBreakV-28K can serve as a comprehensive jailbreak benchmark for MLLMs. <img src="dataset.png" width="800"/> ## 🏆 Mini-Leaderboard | Model | Total ASR| Transfer Attack ASR| |----------------------------|:---------:|:------------------:| | OmniLMM-12B | 58.1 | 70.2 | | InfiMM-Zephyr-7B | 52.9 | 73.0 | | LLaMA-Adapter-v2 | 51.2 | 68.1 | | LLaVA-1.5-13B | 51.0 | 65.5 | | LLaVA-1.5-7B | 46.8 | 61.4 | | InstructBLIP-13B | 45.2 | 55.5 | | InternLM-XComposer2-VL-7B | 39.1 | 29.3 | | Bunny-v1 | 38.0 | 49.5 | | Qwen-VL-Chat | 33.7 | 41.2 | | InstructBLIP-7B | 26.0 | 46.8 | ## ❌ Disclaimers This dataset contains offensive content that may be disturbing, This benchmark is provided for educational and research purposes only. ## 📲 Contact - Weidi Luo: [e-mail] - Siyuan Ma: [e-mail] - Xiaogeng Liu: [e-mail] - Chaowei Xiao: [e-mail] ## 📖 BibTeX: ```python @misc{luo2024jailbreakv28k, title={JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks}, author={Weidi Luo and Siyuan Ma and Xiaogeng Liu and Xiaoyu Guo and Chaowei Xiao}, year={2024}, eprint={2404.03027}, archivePrefix={arXiv}, primaryClass={cs.CR} } ``` [More Information Needed]--- 许可证：MIT 任务类别： - 文本生成 - 问答 友好名称：JailBreakV_28K 规模类别：10K < n < 100K 配置项： - 配置名称：JailBreakV_28K 数据文件： - 拆分：mini_JailBreakV_28K，路径：JailBreakV_28K/mini_JailBreakV_28K.csv - 拆分：JailBreakV_28K，路径：JailBreakV_28K/JailBreakV_28K.csv - 配置名称：RedTeam_2K 数据文件： - 拆分：RedTeam_2K，路径：JailBreakV_28K/RedTeam_2K.csv --- # ⛓‍💥 JailBreakV-28K：评估多模态大语言模型（Multimodal Large Language Model, MLLM）抵御越狱攻击鲁棒性的基准数据集 [**🌐 GitHub 仓库**](https://github.com/EddyLuo1232/JailBreakV_28K) | [**🛎 项目主页**](https://eddyluo1232.github.io/JailBreakV28K/) ｜ [**👉 下载完整数据集**](https://forms.gle/e6boFoM9mNfPHb9H9) ## 如果您喜爱我们的项目，请在 Hugging Face 上为我们点亮星标 ⭐ 以获取最新更新。 ## 📰 动态更新 | 日期 | 事件 | |------------|----------| | **2024/07/09** | 🎉 我们的论文被 COLM 2024 收录。| | **2024/06/22** | 🛠️ 我们已将项目更新至 V0.2 版本，支持用户自定义攻击模型并开展模型评估。| | **2024/04/04** | 🎁 我们的论文已发布至 Arxiv 预印平台。| | **2024/04/03** | 🎉 我们已发布评估与推理示例。| | **2024/03/30** | 🔥 我们已发布数据集。| ## 📥 通过 Hugging Face 数据集库使用本数据集 python from datasets import load_dataset mini_JailBreakV_28K = load_dataset("JailBreakV-28K/JailBreakV-28k", 'JailBreakV_28K')["mini_JailBreakV_28K"] JailBreakV_28K = load_dataset("JailBreakV-28K/JailBreakV-28k", 'JailBreakV_28K')["JailBreakV_28K"] RedTeam_2K = load_dataset("JailBreakV-28K/JailBreakV-28k", 'RedTeam_2K')["RedTeam_2K"] ## 👻 推理与评估 ### 创建运行环境 python conda create -n jbv python=3.9 conda activate jbv pip install -r requirements.txt ### 对多模态大语言模型执行越狱攻击 python # 我们默认使用 Bunny-v1_0，您可通过编辑注释代码将默认攻击模型更换为自定义攻击模型。 # 您可参考 <attack_models> 目录下的 Bunny 脚本添加其他攻击模型。 python attack.py --root JailBreakV_28K ### 开展模型评估 python # 我们默认使用 LlamaGuard，您可通过编辑注释代码将默认评估模型更换为自定义评估模型。 # 您可参考 <evaluate_models> 目录下的 LlamaGuard 脚本添加其他评估模型。 python eval.py --data_path ./results/JailBreakV_28k/<your customized attack model>/JailBreakV_28K.csv ## 😃 数据集详情 JailBreakV_28K 与 mini_JailBreakV_28K 数据集包含以下字段： - **id**：所有样本的唯一标识符。 - **jailbreak_query**：通过各类越狱攻击生成的越狱查询。 - **redteam_query**：RedTeam_2K 中的有害查询。 - **format**：越狱攻击方法，涵盖模板式、说服式、逻辑式、FigStep、查询相关式。 - **policy**：redteam_query 所违反的安全策略。 - **image_path**：图像文件路径。 - **from**：数据来源。 - **selected_mini**：若样本属于 mini_JailBreakV_28K 数据集则为 "True"，否则为 "False"。 - **transfer_from_llm**：若越狱查询源自大语言模型（Large Language Model, LLM）越狱攻击的迁移则为 "True"，否则为 "False"。 RedTeam_2K 数据集包含以下字段： - **id**：所有样本的唯一标识符。 - **question**：有害查询。 - **policy**：有害查询所违反的安全策略。 - **from**：数据来源。 ## 🚀 数据构成 <!-- 提供数据集的简要概述 --> **RedTeam-2K**：RedTeam-2K 是经过精心整理的 2000 条有害查询集合，旨在识别大语言模型与多模态大语言模型的对齐漏洞。该数据集涵盖 16 项安全策略，整合了来自 8 个不同来源的查询。 <img src="distribution_used.png" width="800"/> **JailBreakV-28K**：JailBreakV-28K 包含 28000 条越狱文本-图像对，其中 20000 条为基于文本的大语言模型迁移式越狱攻击样本，8000 条为基于图像的多模态大语言模型越狱攻击样本。该数据集覆盖 16 项安全策略与 5 种多样化的越狱方法。 ## 🛠️ 数据集概览 RedTeam-2K 数据集是经过精心整理的 2000 条有害查询集合，用于识别大语言模型与多模态大语言模型的对齐漏洞。该数据集涵盖 16 项安全策略，整合了来自 8 个不同来源的查询，包括 GPT Rewrite、手工构造（Handcraft）、GPT 生成、LLM 越狱研究、AdvBench、BeaverTails、问题集（Question Set）以及 Anthropic 的 hh-rlhf。 基于 RedTeam-2K 提供的有害查询数据集，JailBreakV-28K 被设计为一个全面且多样化的基准数据集，用于评估越狱攻击从大语言模型向多模态大语言模型的迁移性，以及多模态大语言模型对此类攻击的对齐鲁棒性。具体而言，JailBreakV-28K 包含 28000 条越狱文本-图像对，其中 20000 条为基于文本的大语言模型迁移式越狱攻击样本，8000 条为基于图像的多模态大语言模型越狱攻击样本。该数据集覆盖 16 项安全策略与 5 种多样化的越狱方法。这5种越狱方法分为三类大语言模型迁移攻击：逻辑式（认知过载）、说服式（说服性对抗提示）、模板式（包含贪婪坐标梯度与手工构造策略），以及两类多模态大语言模型攻击：FigStep 攻击与查询相关式攻击。JailBreakV-28K 涵盖了广泛的攻击方法论，并整合了多种图像类型，包括自然图像、随机噪声图像、排版图像、稳定扩散（Stable Diffusion, SD）生成图像、空白图像以及 SD+排版图像。我们认为 JailBreakV-28K 可作为多模态大语言模型的综合性越狱攻击基准数据集。 <img src="dataset.png" width="800"/> ## 🏆 迷你排行榜 | 模型 | 总攻击成功率（ASR）| 迁移攻击成功率（ASR）| |----------------------------|:---------:|:------------------:| | OmniLMM-12B | 58.1 | 70.2 | | InfiMM-Zephyr-7B | 52.9 | 73.0 | | LLaMA-Adapter-v2 | 51.2 | 68.1 | | LLaVA-1.5-13B | 51.0 | 65.5 | | LLaVA-1.5-7B | 46.8 | 61.4 | | InstructBLIP-13B | 45.2 | 55.5 | | InternLM-XComposer2-VL-7B | 39.1 | 29.3 | | Bunny-v1 | 38.0 | 49.5 | | Qwen-VL-Chat | 33.7 | 41.2 | | InstructBLIP-7B | 26.0 | 46.8 | ## ❌ 免责声明 本数据集包含可能令人不适的冒犯性内容，本基准仅用于教育与研究目的。 ## 📲 联系方式 - Weidi Luo: [e-mail] - Siyuan Ma: [e-mail] - Xiaogeng Liu: [e-mail] - Chaowei Xiao: [e-mail] ## 📖 BibTeX引用： python @misc{luo2024jailbreakv28k, title={JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks}, author={Weidi Luo and Siyuan Ma and Xiaogeng Liu and Xiaoyu Guo and Chaowei Xiao}, year={2024}, eprint={2404.03027}, archivePrefix={arXiv}, primaryClass={cs.CR} } [需补充更多信息]