felfri/dose-response-training-annotations

--- license: apache-2.0 tags: - safety - image-classification - llavaguard - dose-response size_categories: - 1M<n<10M --- # Dose-Response Training Data Safety Annotations Safety annotations for the training datasets used in the dose-response experiment. Contains only annotation metadata indexed by image ID — **no images are included**. ## Dataset Description ~8.7M images from 3 source datasets were annotated for safety using [LlavaGuard-7B](https://huggingface.co/AIML-TUDA/LlavaGuard-v1.2-7B-OV). A second full pass was done with Gemini. These annotations were used to construct training subsets with controlled unsafe content fractions (0%–10%). Additionally, a 10K sample (5K safe + 5K unsafe per LlavaGuard) was cross-evaluated by all 4 safety judges. ## Source Datasets | Dataset | Size | Source | |---------|------|--------| | [lehduong/flux_generated](https://huggingface.co/datasets/lehduong/flux_generated) | ~1.7M | FLUX-generated | | [LucasFang/FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M) | ~6M | FLUX-generated with reasoning | | [brivangl/midjourney-v6-llava](https://huggingface.co/datasets/brivangl/midjourney-v6-llava) | ~1M | Midjourney v6 | ## Structure ``` llavaguard/ # Full LlavaGuard-7B annotations (~8.7M images) shard_0.part_01_of_10.parquet ... shard_7.part_10_of_10.parquet gemini/ # Full Gemini annotations (~8.7M images) shard_0.part_01_of_10.parquet ... shard_7.part_10_of_10.parquet cross_judge/ cross_judge_training_sample.parquet # 10K sample annotated by all 4 evaluators ``` ## Annotation Format ### LlavaGuard / Gemini (full annotations) Each parquet file is indexed by image ID and contains columns: - `rating`: Safe / Unsafe - `category`: O1–O9 safety category or "NA: None applying" - `rationale`: Free-text explanation ### Cross-Judge Sample (unified format) | Column | Type | Description | |--------|------|-------------| | `llavaguard_unsafe` | int | 0 = safe, 1 = unsafe (LlavaGuard-7B) | | `llavaguard_category` | str | Safety category (e.g. "O3: Sexual Content") | | `llamaguard3_unsafe` | int | 0 = safe, 1 = unsafe (LlamaGuard-3-11B-Vision) | | `llamaguard3_category` | str | MLCommons category (e.g. "S12: Sexual Content") | | `shieldgemma_unsafe` | int | 0 = safe, 1 = unsafe (ShieldGemma-2-4B) | | `shieldgemma_category` | str | Primary violated policy ("dangerous", "sexual", "violence", or "NA") | | `shieldgemma_p_dangerous` | float | P(dangerous content) | | `shieldgemma_p_sexual` | float | P(sexual content) | | `shieldgemma_p_violence` | float | P(violent content) | | `sd_safety_checker_unsafe` | int | 0 = safe, 1 = unsafe (SD Safety Checker) | Image IDs encode the source dataset and shard, e.g. `lehduong__flux_generated__train-00232-of-00271__004232`. ## Safety Evaluators | Evaluator | Model | Categories | |-----------|-------|------------| | [LlavaGuard-7B](https://huggingface.co/AIML-TUDA/LlavaGuard-v1.2-7B-OV) | Vision-language safety model | O1–O9 (9 categories) | | [LlamaGuard-3-11B-Vision](https://huggingface.co/meta-llama/Llama-Guard-3-11B-Vision) | Meta's multimodal safety model | S1–S14 (MLCommons taxonomy) | | [ShieldGemma-2-4B](https://huggingface.co/google/shieldgemma-2-4b) | Google's safety classifier | dangerous, sexual, violence | | SD Safety Checker | CompVis CLIP-based NSFW classifier | binary (safe/unsafe) | ## Related Resources - [diffusion_safety](https://github.com/felifri/diffusion_safety) — experiment code - Generated images: [felfri/dose-response-generated-images](https://huggingface.co/datasets/felfri/dose-response-generated-images)许可证：Apache-2.0 标签：安全、图像分类、LlavaGuard、剂量反应（dose-response） 规模类别：100万 < 样本量 < 1000万 # 剂量反应训练数据安全标注（Dose-Response Training Data Safety Annotations） 本数据集为剂量反应实验所用训练集的安全标注数据，仅包含以图像ID为索引的标注元数据——**不包含原始图像**。 ## 数据集描述 来自3个源数据集的约870万张图像已通过[LlavaGuard-7B（LlavaGuard-7B）](https://huggingface.co/AIML-TUDA/LlavaGuard-v1.2-7B-OV)完成安全标注，随后又使用Gemini完成了全量二次标注。上述标注被用于构建不安全内容占比可控（0%~10%）的训练子集。 此外，针对10000个样本（按LlavaGuard-7B标注划分，含5000个安全样本与5000个不安全样本）开展了4名安全标注员的交叉评估。 ## 源数据集 | 数据集名称 | 样本规模 | 数据来源 | |---------|------|--------| | [lehduong/flux_generated](https://huggingface.co/datasets/lehduong/flux_generated) | 约170万 | FLUX生成图像 | | [LucasFang/FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M) | 约600万 | 带推理标注的FLUX生成图像 | | [brivangl/midjourney-v6-llava](https://huggingface.co/datasets/brivangl/midjourney-v6-llava) | 约100万 | Midjourney v6生成图像 | ## 数据结构 llavaguard/ # LlavaGuard-7B全量标注集（约870万张图像） shard_0.part_01_of_10.parquet ... shard_7.part_10_of_10.parquet gemini/ # Gemini全量标注集（约870万张图像） shard_0.part_01_of_10.parquet ... shard_7.part_10_of_10.parquet cross_judge/ cross_judge_training_sample.parquet # 由4名标注员共同标注的10000个样本集 ## 标注格式 ### LlavaGuard / Gemini（全量标注格式） 每个Parquet文件以图像ID为索引，包含以下列： - `rating`: 安全 / 不安全 - `category`: O1~O9安全类别或"NA: 无适用类别" - `rationale`: 自由文本形式的标注理由 ### 交叉评估样本（统一标注格式） | 列名 | 数据类型 | 说明 | |--------|------|-------------| | `llavaguard_unsafe` | 整数 | 0=安全，1=不安全（基于LlavaGuard-7B标注） | | `llavaguard_category` | 字符串 | 安全类别（例如"O3: 色情内容"） | | `llamaguard3_unsafe` | 整数 | 0=安全，1=不安全（基于LlamaGuard-3-11B-Vision标注） | | `llamaguard3_category` | 字符串 | MLCommons分类体系下的类别（例如"S12: 色情内容"） | | `shieldgemma_unsafe` | 整数 | 0=安全，1=不安全（基于ShieldGemma-2-4B标注） | | `shieldgemma_category` | 字符串 | 违反的核心政策类别（"危险内容"、"色情内容"、"暴力内容"或"NA"） | | `shieldgemma_p_dangerous` | 浮点数 | 危险内容的预测概率 | | `shieldgemma_p_sexual` | 浮点数 | 色情内容的预测概率 | | `shieldgemma_p_violence` | 浮点数 | 暴力内容的预测概率 | | `sd_safety_checker_unsafe` | 整数 | 0=安全，1=不安全（基于SD安全检查器（SD Safety Checker）标注） | 图像ID编码了源数据集与分片信息，例如`lehduong__flux_generated__train-00232-of-00271__004232`。 ## 安全标注器 | 标注器 | 模型名称 | 分类体系 | |-----------|-------|------------| | [LlavaGuard-7B（LlavaGuard-7B）](https://huggingface.co/AIML-TUDA/LlavaGuard-v1.2-7B-OV) | 多模态视觉语言安全模型 | O1~O9（共9个类别） | | [LlamaGuard-3-11B-Vision（LlamaGuard-3-11B-Vision）](https://huggingface.co/meta-llama/Llama-Guard-3-11B-Vision) | Meta发布的多模态安全模型 | S1~S14（MLCommons分类体系） | | [ShieldGemma-2-4B（ShieldGemma-2-4B）](https://huggingface.co/google/shieldgemma-2-4b) | Google发布的安全分类器 | 危险内容、色情内容、暴力内容 | | SD Safety Checker（SD Safety Checker） | CompVis基于CLIP的不适宜公开内容（NSFW）分类器 | 二元分类（安全/不安全） | ## 相关资源 - [diffusion_safety](https://github.com/felifri/diffusion_safety) — 本实验的代码仓库 - 生成图像数据集：[felfri/dose-response-generated-images（dose-response-generated-images）](https://huggingface.co/datasets/felfri/dose-response-generated-images)