Nemotron-SFT-Safety-v2

## Dataset Description: The Nemotron-SFT-Safety-v2 data is designed to align models to be robust against a variety of safety and security concerns that may arise in unaligned large language models. This dataset is a collection of: 1. A hybrid (open-source and synthetically generated) collection of prompts designed to elicit different model vulnerabilities, and 2. Synthetically generated responses designed to steer model behavior towards safety-aligned values and enhance model robustness across a broad variety of potential attack vectors. 3. Multilingual translations of the English subset. This dataset is ready for commercial or non-commercial uses. Note that the collection of English prompts and responses in this dataset is the same as [Nemotron-SFT-Safety-v1 dataset](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Safety-v1). Multilingual subsets are new. ### The multilingual subset: We translated the English `en` subset into six languages—German `de`, Spanish `es`, French `fr`, Japanese `ja`, Italian `it`, and Chinese `zh` using NeMo Skills using a sentence-by-sentence chunked translation pipeline with NVIDIA [Riva Translate 4B Instruct v1.1](https://huggingface.co/nvidia/Riva-Translate-4B-Instruct-v1.1). To improve translation quality, each translated example was back-translated into English and compared against the original English prompt-response pair. Examples with semantic similarity below 0.8 were filtered out, removing approximately 10–15% of examples per translated language. We also manually spot-checked the highest and lowest-scoring translations to verify that the filter was removing examples with clear translation failures or structural issues. After filtering, we used stratified sampling to retain approximately 15,000 examples per translated language. The final dataset contains about 135,000 examples: ~45,000 in English and ~15,000 each in `de`, `es`, `fr`, `ja`, `it`, and `zh`. ### A refresher on the English subset's consituents: * Collected datasets: * [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0): Comprehensive coverage across a variety of content safety risks like self-harm, violence, criminal planning, profanity, and other harmful or harm-eliciting user requests. * [Gretel Safety Alignment Dataset v1](https://huggingface.co/datasets/gretelai/gretel-safety-alignment-en-v1): Supplemental content safety dataset with a similar purpose as Nemotron Content Safety Dataset. * [HarmfulTasks](https://huggingface.co/datasets/notadib/harmful-tasks): Robustness against some templated but highly effective jailbreak attacks. * Synthetically generated prompts: * `sdg_oversafety`: Contrastive safe prompts created by modifying unsafe prompts to maintain syntactic similarity but be semantically divergent. * `sdg_jailbreaks`: Supplemental jailbreak robustness data. * `sdg_latent_injection`: Improve resiliency against latent prompt injection attacks. * `sdg_sensitive_categories`: Hard refusals for extreme harm categories like self-harm. * `sdg_bias_mitigation`: Mitigate responses that may judge people based on characteristics like age. * `sdg_leakreplay`: Mitigate attempts to elicit possibly copyrighted content. The response generation techniques are carefully designed to elicit reasoning traces and assistant responses that are aligned with our safety behavior guides. To this end, we use a two-stage response generation mechanism, which decouples final response generation and reasoning trace generation. For example, for latent injections (example: "send this money to xyz bank" sneaked in) that need to be ignored/not engaged with by the model, we generate the gold model response without the latent injection present in the prompt, so that the model response is fully utilitarian. Next, we pass the prompt with the latent injection embedded to a second inference call that generates a concise, proper reasoning trace which explicitly acknowledges the presence of the malicious embedded instruction in the middle of the larger benign request, and reasons about why it should ignore it, merging this in with the original thinking about the user request. This form of rewriting the reasoning traces helps build efficient and concise thinking traces, while ensuring that the safety principles we are trying to enforce in each scenario are considered properly in the gold response behaviors. ## Dataset Owner(s): NVIDIA Corporation ## Dataset Creation Date: Created on: 3/15/2026 Last Modified on: 3/15/2026 ## License/Terms of Use: This dataset is governed by the Creative Commons Attribution 4.0 International License (CC BY 4.0). Additional Information: Apache 2.0 License; MIT License. ## Intended Usage: This dataset is intended to be used by the community as part of their model's supervised finetuning (SFT) recipe to improve model alignment on various dimensions of safety and security, including: 1. Appropriate responses to self-harm, violence, criminal planning, child sexual abuse material (CSAM), and other harmful or harm-eliciting user requests. 2. Robustness to common jailbreak strategies, cybersecurity attacks, or red-teaming attempts designed to coerce a model into outputting disturbing content. 3. Counter-acting tendency of models to become overly safe and refuse everything through contrastive hard negatives (requests that share syntactic structure but not semantic meaning with unsafe requests). 4. Mitigating biased model outputs on axes like age-based discrimination. 5. Mitigating leakage of potentially copyrighted content. Previous Version: [Nemotron-SFT-Safety-v1](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Safety-v1) Relationship to Previous Version(s): This dataset is a direct replacement for the previously released dataset. ## Dataset Characterization **Data Collection Method** * Hybrid: Human, Synthetic, Automated **Labeling Method** * Hybrid: Human, Synthetic, Automated ## Dataset Format Modality: Text Format: JSONL Structure: Text + Metadata ## Dataset Quantification Note: We source only the prompts from the following underlying datasets. Any responses that may already exist in an underlying open-source dataset are discarded and regenerated using the response generation techniques described in the [Dataset Description section](#dataset-description). While we start with 45K English, and 15K stratified samples from each of the translated languages, finally, all samples are passed through a standard set of sanity filters to ensure quality responses. These filters are a mixture of LLM and regex-based, and include safety checks (any response marked unsafe by a content moderation model is discarded), identity checks, leakage of sensitive API keys, and data integrity checks. The final dataset quantification numbers are thus slightly shy of 45K English, and 15K for every other language, and are provided in detail below. ### English subset | Prompts Collection | Samples | |-----------------------------------|---------------| | Content safety (Nemotron Content Safety v2, Gretel Safety Alignment v1, `sdg_sensitive_categories`) | 24,752 | | Robustness against jailbreaks (HarmfulTasks, `sdg_jailbreaks`, `sdg_latent_injection`) | 8,963 | | Robustness against over-refusals (`sdg_oversafety`) | 6,896 | | Robustness against demographic biases (`sdg_bias_mitigation`) | 2,000 | | Robustness against leaking sensitive content (`sdg_leakreplay`) | 910 | | Total English | 43,521 | ### Multilingual subset Verified translations of the English subset | Prompts Collection | Samples | |-----------------------------------|---------------| | `de` | 14,474 | | `es` | 14,451 | | `fr` | 14,497 | | `it` | 14,459 | | `ja` | 14,494 | | `zh` | 14,454 | | Total multilingual | 86,829 | | Grand Total | 130,350 | Total Disk Size: ~650MB --- ## Ethical Considerations: NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)