Nemotron-RL-Ultra-Training-Blends

## Dataset Description: This dataset provides Reinforcement Learning (RL) and Multi-teacher On-Policy Distillation (MOPD) training-data blends used by the public **Nemotron-3-Ultra** post-training recipe. The blends are consumed by the [NeMo RL](https://github.com/NVIDIA-NeMo/RL) training [recipes](https://github.com/NVIDIA-NeMo/RL/blob/ultra-v3/docs/guides/nemotron-3-ultra.md) through the [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym) agent framework, in which each prompt is paired with an agent/environment that returns a verifiable or judge-based reward. Each subset is a separate blend; see the recipe for how the blends are used. The blends mix NVIDIA-released datasets with several external datasets (see **Data Preparation** below). This dataset is ready for commercial or non-commercial uses. ## Dataset Owner(s): NVIDIA Corporation ## Dataset Creation Date: Created on: 2026-05-29 Last Modified on: 2026-06-03 ## License/Terms of Use: This dataset is licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.en), [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en), [ODC-BY 1.0](https://opendatacommons.org/licenses/by/1-0/), [MIT](https://opensource.org/license/mit), and [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). ## Intended Usage: This dataset is intended for researchers and developers post-training large language models with reinforcement learning using the NeMo RL recipes and the NeMo Gym agent framework. See the [Nemotron-3-Ultra training guide](https://github.com/NVIDIA-NeMo/RL/blob/ultra-v3/docs/guides/nemotron-3-ultra.md) for end-to-end instructions. ## Dataset Composition Each subset is a separate blend, composed of the following datasets (percentage of the blend). ### rlvr1 | Component (HF dataset) | Ratio | |---|--:| | [nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-v1) | 20.36% | | [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2) + [SWE-Gym/SWE-Gym](https://huggingface.co/datasets/SWE-Gym/SWE-Gym) | 14.12% | | [nvidia/Nemotron-RL-instruction_following](https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following) | 12.02% | | [nvidia/Nemotron-RL-Math-v4](https://huggingface.co/datasets/nvidia/Nemotron-RL-Math-v4) | 10.86% | | [nvidia/Nemotron-RL-coding-competitive_coding](https://huggingface.co/datasets/nvidia/Nemotron-RL-coding-competitive_coding) | 8.06% | | [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) | 4.87% | | [nvidia/Nemotron-RL-ARC-AGI-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ARC-AGI-v1) | 4.23% | | [nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1) | 4.09% | | [nvidia/Nemotron-RL-QA-Abstention-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-QA-Abstention-v1) | 4.09% | | [nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1) | 4.08% | | [nvidia/Nemotron-RL-Safety-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Safety-v1) | 2.91% | | [nvidia/Nemotron-RL-knowledge-mcqa](https://huggingface.co/datasets/nvidia/Nemotron-RL-knowledge-mcqa) | 2.13% | | [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1) | 2.12% | | [nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2) | 2.11% | | [nvidia/Nemotron-SFT-Science-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Science-v2) | 2.11% | | [nvidia/Nemotron-RL-Instruction-Following-Calendar-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Calendar-v2) | 0.92% | | [nvidia/Nemotron-Math-Proofs-v1](https://huggingface.co/datasets/nvidia/Nemotron-Math-Proofs-v1) | 0.92% | ### rlvr2 | Component (HF dataset) | Ratio | |---|--:| | [nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-v1) | 20.21% | | [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2) + [SWE-Gym/SWE-Gym](https://huggingface.co/datasets/SWE-Gym/SWE-Gym) | 14.02% | | [nvidia/Nemotron-RL-Math-v4](https://huggingface.co/datasets/nvidia/Nemotron-RL-Math-v4) | 10.79% | | [nvidia/Nemotron-RL-instruction_following](https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following) | 10.50% | | [nvidia/Nemotron-RL-coding-competitive_coding](https://huggingface.co/datasets/nvidia/Nemotron-RL-coding-competitive_coding) | 8.00% | | [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) | 4.80% | | [nvidia/Nemotron-RL-ARC-AGI-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ARC-AGI-v1) | 4.20% | | [nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1) | 4.06% | | [nvidia/Nemotron-RL-QA-Abstention-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-QA-Abstention-v1) | 4.06% | | [nvidia/Nemotron-RL-Safety-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Safety-v1) | 2.89% | | [nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1) | 2.69% | | [nvidia/Nemotron-RL-knowledge-mcqa](https://huggingface.co/datasets/nvidia/Nemotron-RL-knowledge-mcqa) | 2.12% | | [nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2) | 2.10% | | [nvidia/Nemotron-SFT-Science-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Science-v2) | 2.10% | | [nvidia/Nemotron-RL-Litmus-Bench-v0.1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Litmus-Bench-v0.1) | 1.42% | | [nvidia/Nemotron-RL-Instruction-Following-Citation-Formatting-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Citation-Formatting-v1) | 1.42% | | [nvidia/Nemotron-RL-Instruction-Following-Free-Form-Formatting-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Free-Form-Formatting-v1) | 1.40% | | [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1) | 1.40% | | [nvidia/Nemotron-RL-Instruction-Following-Calendar-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Calendar-v2) | 0.92% | | [nvidia/Nemotron-Math-Proofs-v1](https://huggingface.co/datasets/nvidia/Nemotron-Math-Proofs-v1) | 0.91% | ### ifbench | Component (HF dataset) | Ratio | |---|--:| | [nvidia/Nemotron-RL-QA-Abstention-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-QA-Abstention-v1) | 43.00% | | [nvidia/Nemotron-RL-instruction_following](https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following) | 21.46% | | [nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1) | 21.18% | | [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) | 7.19% | | [nvidia/Nemotron-RL-InverseIFEval-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-InverseIFEval-v1) | 7.17% | ### rlhf | Component (HF dataset) | Ratio | |---|--:| | [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) | 84.62% | | [nvidia/Nemotron-RL-Safety-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Safety-v1) | 15.38% | ### reasoning | Component (HF dataset) | Ratio | |---|--:| | [virtuoussy/Multi-subject-RLVR](https://huggingface.co/datasets/virtuoussy/Multi-subject-RLVR) | 100.00% | ### swe | Component (HF dataset) | Ratio | |---|--:| | [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2) | 97.36% | | [SWE-Gym/SWE-Gym](https://huggingface.co/datasets/SWE-Gym/SWE-Gym) | 2.64% | ### mopd | Component (HF dataset) | Ratio | |---|--:| | [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2) + [SWE-Gym/SWE-Gym](https://huggingface.co/datasets/SWE-Gym/SWE-Gym) | 15.27% | | [nvidia/Nemotron-RL-Math-v4](https://huggingface.co/datasets/nvidia/Nemotron-RL-Math-v4) | 12.44% | | [nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2) | 12.42% | | [nvidia/Nemotron-RL-instruction_following](https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following) | 12.11% | | [nvidia/Nemotron-RL-coding-competitive_coding](https://huggingface.co/datasets/nvidia/Nemotron-RL-coding-competitive_coding) | 9.22% | | [nvidia/Nemotron-RL-ARC-AGI-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ARC-AGI-v1) | 4.85% | | [nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1) | 4.68% | | [nvidia/Nemotron-RL-QA-Abstention-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-QA-Abstention-v1) | 4.68% | | [nvidia/Nemotron-RL-Safety-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Safety-v1) | 4.06% | | [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) | 4.05% | | [nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1) | 3.10% | | [nvidia/Nemotron-RL-knowledge-mcqa](https://huggingface.co/datasets/nvidia/Nemotron-RL-knowledge-mcqa) | 2.44% | | [nvidia/Nemotron-SFT-Science-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Science-v2) | 2.42% | | [nvidia/Nemotron-RL-Agentic-Indirect-Prompt-Injection-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Indirect-Prompt-Injection-v1) | 2.33% | | [nvidia/Nemotron-RL-Litmus-Bench-v0.1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Litmus-Bench-v0.1) | 1.63% | | [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1) | 1.61% | | [nvidia/Nemotron-RL-Instruction-Following-Calendar-v2](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Calendar-v2) | 1.06% | | [nvidia/Nemotron-Math-Proofs-v1](https://huggingface.co/datasets/nvidia/Nemotron-Math-Proofs-v1) | 1.05% | | [nvidia/Nemotron-RL-Instruction-Following-Citation-Formatting-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Citation-Formatting-v1) | 0.29% | | [nvidia/Nemotron-RL-Instruction-Following-Free-Form-Formatting-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Free-Form-Formatting-v1) | 0.29% | ## Data Preparation For the BytedTsinghua-SIA/DAPO-Math-17k and Skywork/Skywork-OR1-RL-Data data in the blends, instead of replicating the data directly, placeholders are used that point to entries in the original datasets. Use the [fill_placeholders.py](fill_placeholders.py) script to download the data from the original datasets into the blends. For each dataset a user elects to use, the user is responsible for checking if the dataset license is fit for the intended purpose. For the RLHF-GenRM data, this release currently includes only data from the previously released [nvidia/Nemotron-RLHF-GenRM-v1](https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1) dataset. We plan to release `nvidia/Nemotron-RLHF-GenRM-v2` with updated data shortly, after which these blends will be updated to include the additional data. Each dataset blend is preprocessed according to the curriculum described in the [Nemotron-3-Ultra technical report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf). Samples are ordered from higher pass-rate (easier) to lower pass-rate (harder), ensuring a balanced learning progression. ## Dataset Characterization **Data Collection Method** * Hybrid: Human, Synthetic **Labeling Method** * Hybrid: Human, Synthetic, Automated ## Dataset Format Modality: Text Format: JSONL (NeMo Gym prompt/agent records) Structure: Text + Metadata ## Dataset Quantification | Subset | Samples | Size | |----------------|--------:|-----:| | rlvr1 | 98,424 | 5.0 GB | | rlvr2 | 99,116 | 5.0 GB | | ifbench | 34,649 | 890 MB | | rlhf | 6,500 | 36 MB | | reasoning | 5,236 | 4.5 MB | | swe | 7,816 | 563 MB | | mopd | 85,980 | 5.5 GB | | **Total** | **337,721** | **~17.1 GB** | ## Reference(s): - NeMo RL: https://github.com/NVIDIA-NeMo/RL - NeMo Gym: https://github.com/NVIDIA-NeMo/Gym - Nemotron-3-Ultra training guide: https://github.com/NVIDIA-NeMo/RL/blob/ultra-v3/docs/guides/nemotron-3-ultra.md ## Ethical Considerations: NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)