Nemotron-Content-Safety-Audio-Dataset

# Nemotron Content Safety Audio Dataset

## Dataset Description

The **Nemotron Content Safety Audio Dataset** is a multimodal extension of the [Nemotron Content Safety Dataset V2 (Aegis 2.0)](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0), comprising **1,928 audio files** generated from the test set prompts. This dataset enables multimodal AI safety research by providing spoken versions of adversarial and safety-critical prompts across 23 violation categories.

**LANGUAGE:** All prompts are in English. However, the audio files were generated using NVIDIA's Magpie-Multilingual Text-to-Speech (TTS) model with speakers of different native language backgrounds, providing accent and dialect diversity.

**Voice Distribution by Native Language Background:**
- **English native speakers (EN-US)**: 648 files (33.6%) - 2 voices (Ray, Sofia)
- **Spanish native speakers (ES-US)**: 642 files (33.3%) - 2 voices (Diego, Isabela)
- **French native speakers (FR-FR)**: 638 files (33.1%) - 2 voices (Louise, Pascal)

Each audio file corresponds to a text prompt from the original Aegis 2.0 test set, maintaining the original human-annotated safety labels and violation categories. The dataset includes both safe (46.1%) and unsafe (53.9%) content, with prompts ranging from 1 to 5,499 characters in length.

**This dataset is ready for commercial use.**

## Dataset Owner

**NVIDIA Corporation**

**Created:** November 21, 2025

**License:** CC-BY-4.0 (Creative Commons Attribution 4.0 International)

Full license text: [https://creativecommons.org/licenses/by/4.0/legalcode](https://creativecommons.org/licenses/by/4.0/legalcode)

## Intended Usage

### Primary Use Cases

1. **Multimodal AI Safety Research** - Developing guardrails that work across text and audio modalities
2. **Audio Content Moderation** - Training models to detect unsafe content in spoken form
3. **Voice-based Red-teaming** - Evaluating LLM safety when receiving audio input
4. **Adversarial Audio Testing** - Assessing how safety systems respond to spoken adversarial prompts
5. **Cross-modal Safety Alignment** - Building systems that maintain safety across text-to-speech and speech-to-text pipelines

### Target Users

- AI Safety researchers
- Content moderation system developers
- Multimodal model developers
- Guardrail system engineers
- Academic researchers in AI ethics and alignment

### Out-of-Scope Use

- Training conversational agents or dialogue systems (will likely lead to harmful behavior)
- Any use that violates the CC-BY-4.0 license terms

## Dataset Characterization

**Data Collection Method:**
- **Hybrid: Human, Synthetic**
- Original prompts: Human-written, collected from Anthropic RLHF, DAN jailbreak dataset, and AI-assisted Red-Teaming datasets
- Audio generation: Synthetic, generated using NVIDIA Magpie-Multilingual TTS model
- Text chunking strategy: Automated splitting (max 300 chars per chunk) with sentence-aware boundaries
- Voice assignment: Automated round-robin distribution for balanced voice usage

**Labeling Method:**
- **Human** - All prompt safety labels are human-annotated
- Binary safety labels (safe/unsafe)
- Multi-category violation tags (23 categories)
- Quality assured through systematic review process (15-30% of data re-evaluated)
- Annotator team: 12 annotators with Engineering and Creative Writing backgrounds
- **Note:** Audio files inherit labels from original text prompts; no separate audio labeling performed

## Dataset Format

**Modality:** Audio (WAV format) + Metadata (CSV)

**Format Classification:**
- **Audio Files:**
- Format: Uncompressed WAV (Waveform Audio File Format)
- Language: English (all content)
- Speaker Diversity: Native English, Spanish, and French speakers (accents)
- Sample Rate: 44,100 Hz
- Bit Depth: 16-bit
- Channels: Mono
- Encoding: PCM (Pulse Code Modulation)

- **Metadata File:** `metadata.csv`
- Format: CSV (Comma-Separated Values)
- Encoding: UTF-8
- Columns: 12 (id, response, prompt_label, response_label, violated_categories, prompt_label_source, response_label_source, prompt, audio_filename, audio_duration_seconds, speaker_name, speaker_native_language)

**File Naming Convention:**
- Pattern: `{id}_{voice_name}.wav`
- Example: `3e26487c2fac48cb9297f043e3b0238d_Magpie-Multilingual.EN-US.Mia.wav`
- ID corresponds to the `id` column in the metadata CSV

**Dataset Structure:**
```
nemotron-content-safety-audio/
├── audio/ # 1,928 WAV files
│ └── {id}_{voice}.wav
├── metadata.csv # Complete dataset metadata
├── README.md # Dataset documentation
└── LICENSE # CC-BY-4.0 license
```

## Dataset Quantification

**Record Count:**
- **Total Audio Files:** 1,928
- **Safe Prompts:** 889 audio files (46.1%)
- **Unsafe Prompts:** 1,039 audio files (53.9%)

**Feature Count:**
- **Audio Features:** Duration, sample rate, voice identity, language
- **Metadata Features (per record):**
- Unique ID
- Original text prompt
- LLM response (where available)
- Binary safety labels (prompt and response)
- Violated categories (23 unique categories)
- Label sources (human/llm_jury)

**Length Distribution:**
- **Prompt Text Length:**
- Min: 1 character
- Max: 5,499 characters
- Mean: 262.0 characters
- Median: 67 characters
- Distribution:
- Very Short (0-100 chars): 1,314 files (68.2%)
- Medium (101-500 chars): 407 files (21.1%)
- Long (501-1000 chars): 74 files (3.8%)
- Very Long (1001-5000 chars): 130 files (6.7%)
- Extremely Long (5000+ chars): 3 files (0.2%)

**Voice Distribution by Speaker Native Language Background:**
- **English native speakers (EN-US):** 648 files (33.6%)
- Ray: 324 files (16.8%)
- Sofia: 324 files (16.8%)
- **Spanish native speakers (ES-US):** 642 files (33.3%)
- Diego: 322 files (16.7%)
- Isabela: 320 files (16.6%)
- **French native speakers (FR-FR):** 638 files (33.1%)
- Louise: 317 files (16.4%)
- Pascal: 321 files (16.6%)

*Note: All audio content is spoken in English; voice categories reflect the native language background of each synthetic speaker, providing accent and pronunciation diversity.*

**Violation Categories Distribution (Top 10):**
1. Criminal Planning/Confessions: 509 prompts (26.4%)
2. Needs Caution: 235 prompts (12.2%)
3. Hate/Identity Hate: 182 prompts (9.4%)
4. Violence: 178 prompts (9.2%)
5. Harassment: 152 prompts (7.9%)
6. Controlled/Regulated Substances: 114 prompts (5.9%)
7. PII/Privacy: 101 prompts (5.2%)
8. Profanity: 100 prompts (5.2%)
9. Sexual: 99 prompts (5.1%)
10. Guns and Illegal Weapons: 62 prompts (3.2%)

**Multi-Category Violations:**
- 511 prompts (26.5%) have multiple category violations
- Most common combinations:
- Controlled Substances + Criminal Planning: 62 prompts
- Criminal Planning + Violence: 51 prompts
- Harassment + Hate: 23 prompts
- Criminal Planning + PII/Privacy: 21 prompts
- Criminal Planning + Needs Caution: 21 prompts

**Total Data Storage:**
- **Audio Files:** Approximately 2.7 GB (average ~1.4 MB per file)
- **Metadata CSV:** 1.3 MB
- **Total Dataset Size:** ~2.7 GB

**Content Characteristics:**
- First-person prompts (I/me/my): 1,152 (59.8%)
- Questions (?): 1,019 (52.9%)
- Multi-sentence prompts: 976 (50.6%)
- Contains URLs: 23 (1.2%)
- Casual/slang language: 41 (2.1%)

### Data Annotation Ethics
The original text dataset annotation process adhered to strict ethical standards:
- **Voluntary Participation**: All annotators volunteered after understanding the nature of potentially toxic content.
- **Mental Health Support**: Regular 1:1 meetings, Adult Content Acknowledgement forms, and wellness resources provided.
- **Bias Prevention**: Label Studio tooling prevented annotators from seeing each other's work.
- **Quality Assurance**: 15-30% of annotations re-evaluated by project leads with feedback loops.
- **Diverse Team**: 12 annotators from Engineering and Creative Writing backgrounds, representing various ethnicities, religions, ages, and social backgrounds.

### Audio Generation Ethics

- **Synthetic Voices**: All audio uses synthetic TTS voices to avoid exploiting human voice actors for unsafe content.
- **No Human Speakers**: Protects individuals from having to vocalize harmful, offensive, or traumatic content.
- **Voice Diversity**: Balanced gender and language representation across voice models.
- **Transparent Attribution**: Clear documentation that audio is TTS-generated, not human speech.

### Responsible Use Guidelines

1. Do not use this dataset to train conversational agents without extensive safety alignment
2. Implement content warnings and access controls when sharing or demonstrating
3. Consider the mental health impact on researchers working with this content
4. Report any misuse or concerns about dataset content
5. Ensure compliance with local laws regarding offensive content
6. Do not attempt to use audio files to harm, harass, or deceive individuals

## Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Citation

If you use this dataset, please cite both the audio dataset and the original Aegis 2.0 paper:

```bibtex
@dataset{nemotron_audio_safety_2025,
title = {Nemotron Content Safety Audio Dataset},
author = {Hazare, Akshay},
year = {2025},
month = {December},
publisher = {NVIDIA Corporation},
url = {https://huggingface.co/datasets/nvidia/Nemotron-Content-Safety-Audio-Dataset},
license = {CC-BY-4.0}
}

@inproceedings{ghosh-etal-2025-aegis2,
title = "{AEGIS}2.0: A Diverse {AI} Safety Dataset and Risks Taxonomy for Alignment of {LLM} Guardrails",
author = "Ghosh, Shaona and Varshney, Prasoon and Sreedhar, Makesh Narsimhan and Padmakumar, Aishwarya and Rebedea, Traian and Varghese, Jibin Rajan and Parisien, Christopher",
editor = "Chiruzzo, Luis and Ritter, Alan and Wang, Lu",
booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.naacl-long.306/",
doi = "10.18653/v1/2025.naacl-long.306",
pages = "5992--6026",
ISBN = "979-8-89176-189-6"
}
```

## Related Resources

- **Original Text Dataset:** [nvidia/Aegis-AI-Content-Safety-Dataset-2.0](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0)
- **Paper:** [AEGIS 2.0 on ACL Anthology](https://aclanthology.org/2025.naacl-long.306/) | [arXiv](https://arxiv.org/abs/2501.09004)
- **NVIDIA Guardrails:** [NeMo Guardrails](https://github.com/NVIDIA/NeMo-Guardrails)
- **TTS Model:**
NVIDIA Magpie-Multilingual
Function ID: 877104f7-e885-42b9-8de8-f6e4c6303969
API: NVIDIA Cloud Functions (NVCF)

## Dataset Card Author(s)

**Audio Dataset:**
Akshay Hazare ([e-mail])

**Original Dataset:**
Shaona Ghosh, Prasoon Varshney ({shaonag, prasoonv}@nvidia.com)

## Acknowledgments

- NeMo Guardrails team, NVIDIA
- NVIDIA Magpie team (TTS service)
- 12 human annotators with diverse backgrounds
- Data engineering and quality assurance teams

## Version History

- **v1.0** (December 1, 2025): Initial release with 1,928 audio files

---

# Nemotron内容安全音频数据集

## 数据集描述
**Nemotron内容安全音频数据集**是[Nemotron内容安全数据集V2（Aegis 2.0）](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0)的多模态扩展版本，包含从其测试集提示生成的**1928个音频文件**。本数据集为多模态AI安全研究提供支持，涵盖23个违规类别，包含对抗性与安全关键型提示的语音版本。

**语言**：所有提示均为英语。但音频文件由NVIDIA的Magpie-Multilingual文本转语音（Text-to-Speech, TTS）模型生成，使用了不同母语背景的发音人，具备口音与方言多样性。

**按母语背景划分的发音人分布**：
- **英语母语发音人（EN-US）**：648个文件（占比33.6%），对应2位发音人（Ray、Sofia）
- **西班牙语母语发音人（ES-US）**：642个文件（占比33.3%），对应2位发音人（Diego、Isabela）
- **法语母语发音人（FR-FR）**：638个文件（占比33.1%），对应2位发音人（Louise、Pascal）

每个音频文件对应原始Aegis 2.0测试集中的文本提示，保留了原始人工标注的安全标签与违规类别。数据集包含安全内容（占比46.1%）与不安全内容（占比53.9%），提示长度介于1至5499个字符之间。
**本数据集可用于商业用途**。

## 数据集所有者
**NVIDIA公司（NVIDIA Corporation）**
**创建时间**：2025年11月21日
**许可证**：知识共享署名4.0国际许可协议（Creative Commons Attribution 4.0 International, CC-BY-4.0）
完整许可证文本：[https://creativecommons.org/licenses/by/4.0/legalcode](https://creativecommons.org/licenses/by/4.0/legalcode)

## 预期用途
### 主要应用场景
1. **多模态AI安全研究**：开发可跨文本与音频模态生效的安全护栏
2. **音频内容审核**：训练模型检测语音形式的不安全内容
3. **基于语音的红队测试**：评估大语言模型（LLM）在接收音频输入时的安全性
4. **对抗性音频测试**：评估安全系统对语音对抗性提示的响应能力
5. **跨模态安全对齐**：构建可在文本转语音、语音转文本 pipeline 中保持安全性能的系统

### 目标用户
- AI安全研究人员
- 内容审核系统开发者
- 多模态模型开发者
- 安全护栏系统工程师
- AI伦理与对齐领域的学术研究者

### 超出范围的用途
- 训练对话智能体或对话系统（可能导致有害行为）
- 任何违反CC-BY-4.0许可证条款的使用

## 数据集特征描述
**数据收集方式**：
- **混合模式：人工+合成**
- 原始提示：人工撰写，收集自Anthropic人类反馈强化学习（Reinforcement Learning from Human Feedback, RLHF）、DAN越狱数据集以及AI辅助红队测试数据集
- 音频生成：合成生成，使用NVIDIA Magpie多语言TTS模型
- 文本分块策略：自动分块（每块最大300字符），且以句子为边界进行分割
- 发音人分配：采用自动轮询分配方式，确保发音人使用均衡

**标注方式**：
- **人工标注**：所有提示的安全标签均为人工标注
- 二元安全标签（安全/不安全）
- 多类别违规标签（共23个类别）
- 经过系统评审流程确保标注质量（15%-30%的数据会被重新评估）
- 标注团队：12位具备工程与创意写作背景的标注人员
- **注**：音频文件继承自原始文本提示的标签，未对音频进行单独标注

## 数据集格式
**模态**：音频（波形音频文件格式，Waveform Audio File Format, WAV）+ 元数据（逗号分隔值，Comma-Separated Values, CSV）
**格式分类**：
- **音频文件**：
- 格式：未压缩WAV
- 语言：英语（所有内容）
- 发音人多样性：英语、西班牙语、法语母语发音人（带不同口音）
- 采样率：44100 Hz
- 位深度：16-bit
- 声道：单声道
- 编码：脉冲编码调制（Pulse Code Modulation, PCM）
- **元数据文件**：`metadata.csv`
- 格式：CSV
- 编码：UTF-8
- 列数：12列，分别为：id、response、prompt_label、response_label、violated_categories、prompt_label_source、response_label_source、prompt、audio_filename、audio_duration_seconds、speaker_name、speaker_native_language
**文件命名规则**：
- 格式：`{id}_{voice_name}.wav`
- 示例：`3e26487c2fac48cb9297f043e3b0238d_Magpie-Multilingual.EN-US.Mia.wav`
- ID对应元数据CSV中的`id`列
**数据集目录结构**：

nemotron-content-safety-audio/
├── audio/ # 1928个WAV音频文件
│ └── {id}_{voice}.wav
├── metadata.csv # 完整数据集元数据
├── README.md # 数据集文档
└── LICENSE # CC-BY-4.0许可证文件

## 数据集量化统计
**样本数量**：
- **总音频文件数**：1928
- **安全提示对应音频文件**：889个（占比46.1%）
- **不安全提示对应音频文件**：1039个（占比53.9%）
**特征维度**：
- **音频特征**：时长、采样率、发音人身份、语言
- **单条记录元数据特征**：
- 唯一ID
- 原始文本提示
- 大语言模型（LLM）生成的回复（如适用）
- 二元安全标签（提示与回复）
- 违规类别（共23个唯一类别）
- 标签来源（人工/LLM评审）
**提示文本长度分布**：
- **提示文本长度**：
- 最小值：1字符
- 最大值：5499字符
- 平均值：262.0字符
- 中位数：67字符
- 分布情况：
- 极短（0-100字符）：1314个文件（占比68.2%）
- 中等（101-500字符）：407个文件（占比21.1%）
- 长文本（501-1000字符）：74个文件（占比3.8%）
- 超长文本（1001-5000字符）：130个文件（占比6.7%）
- 极长文本（5000+字符）：3个文件（占比0.2%）
**按母语背景划分的发音人分布**：
- **英语母语发音人（EN-US）**：648个文件（占比33.6%）
- Ray：324个文件（占比16.8%）
- Sofia：324个文件（占比16.8%）
- **西班牙语母语发音人（ES-US）**：642个文件（占比33.3%）
- Diego：322个文件（占比16.7%）
- Isabela：320个文件（占比16.6%）
- **法语母语发音人（FR-FR）**：638个文件（占比33.1%）
- Louise：317个文件（占比16.4%）
- Pascal：321个文件（占比16.6%）
*注：所有音频内容均为英语发音；发音人类别反映合成发音人的母语背景，以此提供口音与发音多样性。*
**违规类别分布（前10名）**：
1. 犯罪策划/供认：509个提示（占比26.4%）
2. 需要谨慎处理：235个提示（占比12.2%）
3. 仇恨言论/身份仇恨：182个提示（占比9.4%）
4. 暴力内容：178个提示（占比9.2%）
5. 骚扰：152个提示（占比7.9%）
6. 受管制物质：114个提示（占比5.9%）
7. 个人可识别信息（Personally Identifiable Information, PII）/隐私：101个提示（占比5.2%）
8. 亵渎用语：100个提示（占比5.2%）
9. 色情内容：99个提示（占比5.1%）
10. 枪支与非法武器：62个提示（占比3.2%）
**多类别违规情况**：
- 511个提示（占比26.5%）存在多个类别违规
- 最常见的组合：
- 受管制物质 + 犯罪策划：62个提示
- 犯罪策划 + 暴力内容：51个提示
- 骚扰 + 仇恨言论：23个提示
- 犯罪策划 + PII/隐私：21个提示
- 犯罪策划 + 需要谨慎处理：21个提示
**总数据存储量**：
- **音频文件**：约2.7 GB（平均每个文件约1.4 MB）
- **元数据CSV**：1.3 MB
- **数据集总大小**：约2.7 GB
**内容特征**：
- 第一人称提示（使用I/me/my）：1152个（占比59.8%）
- 疑问句（包含?）：1019个（占比52.9%）
- 多句提示：976个（占比50.6%）
- 包含URL：23个（占比1.2%）
- 使用口语/俚语：41个（占比2.1%）

### 数据标注伦理
原始文本数据集的标注流程遵循严格的伦理标准：
- **自愿参与**：所有标注人员均在了解潜在有害内容的性质后自愿参与。
- **心理健康支持**：提供定期一对一会谈、成人内容知情同意书以及心理健康资源。
- **偏见防控**：使用Label Studio工具避免标注人员查看彼此的标注结果。
- **质量保障**：15%-30%的标注内容由项目负责人重新评估，并建立反馈循环。
- **团队多样性**：12位标注人员具备工程与创意写作背景，涵盖不同种族、宗教、年龄与社会背景。

### 音频生成伦理
- **合成发音人**：所有音频均使用合成TTS发音人生成，避免利用真人发音人制作有害内容。
- **无真人发声**：保护个体无需口述有害、冒犯性或创伤性内容。
- **发音人多样性**：在发音人模型中实现均衡的性别与语言代表性。
- **透明归因**：明确标注音频为TTS生成内容，而非真人语音。

### 负责任使用指南
1. 未经充分的安全对齐训练，不得使用本数据集训练对话智能体
2. 分享或演示本数据集内容时，应添加内容警告与访问控制
3. 考虑研究人员处理此类内容时的心理健康影响
4. 报告任何误用或对数据集内容的担忧
5. 确保遵守当地关于冒犯性内容的法律法规
6. 不得使用音频文件伤害、骚扰或欺骗他人

## 伦理考量
NVIDIA认为可信AI是共同责任，我们已制定相关政策与实践，以支持各类AI应用的开发。开发者在按照服务条款下载或使用本数据集时，应与内部模型团队协作，确保本数据集符合相关行业与应用场景的要求，并应对未预见的产品误用问题。

请通过[此链接](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)报告模型质量、风险、安全漏洞或NVIDIA AI相关问题。

## 引用说明
若使用本数据集，请同时引用本音频数据集与原始Aegis 2.0论文：
bibtex
@dataset{nemotron_audio_safety_2025,
title = {Nemotron内容安全音频数据集},
author = {Hazare, Akshay},
year = {2025},
month = {December},
publisher = {NVIDIA Corporation},
url = {https://huggingface.co/datasets/nvidia/Nemotron-Content-Safety-Audio-Dataset},
license = {CC-BY-4.0}
}

@inproceedings{ghosh-etal-2025-aegis2,
title = "{AEGIS}2.0：面向大语言模型（LLM）安全护栏对齐的多样化AI安全数据集与风险分类体系",
author = "Ghosh, Shaona and Varshney, Prasoon and Sreedhar, Makesh Narsimhan and Padmakumar, Aishwarya and Rebedea, Traian and Varghese, Jibin Rajan and Parisien, Christopher",
editor = "Chiruzzo, Luis and Ritter, Alan and Wang, Lu",
booktitle = "2025年计算语言学协会美洲分会会议论文集（卷1：长论文）",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.naacl-long.306/",
doi = "10.18653/v1/2025.naacl-long.306",
pages = "5992--6026",
ISBN = "979-8-89176-189-6"
}

## 相关资源
- **原始文本数据集**：[nvidia/Aegis-AI-Content-Safety-Dataset-2.0](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0)
- **论文**：[ACL Anthology上的AEGIS 2.0论文](https://aclanthology.org/2025.naacl-long.306/) | [arXiv预印本](https://arxiv.org/abs/2501.09004)
- **NVIDIA安全护栏**：[NeMo Guardrails](https://github.com/NVIDIA/NeMo-Guardrails)
- **TTS模型**：
NVIDIA Magpie多语言模型
功能ID：877104f7-e885-42b9-8de8-f6e4c6303969
API：NVIDIA云函数（NVIDIA Cloud Functions, NVCF）

## 数据集卡片作者
**音频数据集作者**：
Akshay Hazare ([e-mail])
**原始数据集作者**：
Shaona Ghosh, Prasoon Varshney ({shaonag, prasoonv}@nvidia.com)

## 致谢
- NVIDIA NeMo Guardrails团队
- NVIDIA Magpie团队（TTS服务）
- 12位背景多样的人工标注人员
- 数据工程与质量保障团队

## 版本历史
- **v1.0**（2025年12月1日）：首次发布，包含1928个音频文件

---