Llama-Nemotron-Post-Training-Dataset-v1

# Llama-Nemotron-Post-Training-Dataset-v1.1 Release

**Update [4/8/2025]:**

**v1.1:** We are releasing an additional 2.2M Math and 500K Code Reasoning Data in support of our release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1). 🎉
## Data Overview

This dataset is a compilation of SFT and RL data that supports improvements of math, code, general reasoning, and instruction following capabilities of the original Llama instruct model, in support of NVIDIA’s release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1), [Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1) and [Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1).

Llama-3.1-Nemotron-Ultra-253B-v1 is a large language model (LLM) which is a derivative of [Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) (AKA the *reference model*).
Llama-3.3-Nemotron-Super-49B-v1 is an LLM which is a derivative of [Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) (AKA the *reference model*). Llama-3.1-Nemotron-Nano-8B-v1 is an LLM which is a derivative of [Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) (AKA the *reference model*). They are aligned for human chat preferences, and tasks.

These models offer a great tradeoff between model accuracy and efficiency. Efficiency (throughput) directly translates to savings. Using a novel Neural Architecture Search (NAS) approach, we greatly reduce the model’s memory footprint and enable larger workloads. This NAS approach enables the selection of a desired point in the accuracy-efficiency tradeoff. The models support a context length of 128K.

This dataset release represents a significant move forward in openness and transparency in model development and improvement. By releasing the complete training set, in addition to the training technique, tools and final model weights, NVIDIA supports both the re-creation and the improvement of our approach.

## Data distribution

| Category | Value |
|----------|-----------|
| math | 22,066,397|
| code | 10,108,883 |
| science | 708,920 |
| instruction following | 56,339 |
| chat | 39,792 |
| safety | 31,426 |

## Filtering the data

Users can download subsets of the data based on the metadata schema described above. Example script for downloading code and math as follows:
```
from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])
```

## Prompts

Prompts have been sourced from either public and open corpus or synthetically generated. All responses have been synthetically generated from public and open models.

The prompts were extracted, and then filtered for quality and complexity, or generated to meet quality and complexity requirements. This included filtration such as removing inconsistent prompts, prompts with answers that are easy to guess, and removing prompts with incorrect syntax.

## Responses

Responses were synthetically generated by a variety of models, with some prompts containing responses for both reasoning on and off modes, to train the model to distinguish between two modes.

Models that were used in the creation of this dataset:

| Model | Number of Samples |
|----------|-----------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## License/Terms of Use

The dataset contains information about license type on a per sample basis. The dataset is predominantly CC-BY-4.0, with a small subset of prompts from Wildchat having an ODC-BY license and a small subset of prompts from StackOverflow with CC-BY-SA license.

This dataset contains synthetic data created using Llama-3.3-70B-Instruct, Llama-3.1-Nemotron-70B-Instruct and
Llama-3.3-Nemotron-70B-Feedback/Edit/Select (ITS models). If this dataset is used to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, such AI model may be subject to redistribution and use requirements in the Llama 3.1 Community License Agreement and Llama 3.3 Community License Agreement.

**Data Developer:** NVIDIA

### Use Case: <br>
Developers training AI Agent systems, chatbots, RAG systems, and other AI-powered applications. <br>

### Release Date: <br>
4/8/2025 <br>

## Data Version
1.1 (4/8/2025)

## Intended use

The Llama Nemotron Post-Training Dataset is intended to be used by the community to continue to improve open models. The data may be freely used to train and evaluate.

## Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Data Opt-Out:
NVIDIA has undertaken legal review to ensure there is no confidential, PII or copyright materials. If, when reviewing or using this dataset, you identify issues with the data itself, such as those listed above, please contact [e-mail].

## Citation
```
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}
```

# Llama-Nemotron-Post-Training-Dataset-v1.1 发布版本

**更新 [2025年4月8日]:**

**v1.1:** 我们新增发布220万条数学推理数据与50万条代码推理数据，以配套支持[Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)的发布。🎉

## 数据集概览
本数据集为监督微调(Supervised Fine-Tuning, SFT)与强化学习(Reinforcement Learning, RL)数据的合集，旨在优化原版Llama指令模型的数学、代码、通用推理与指令遵循能力，配套支持NVIDIA发布的[Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)、[Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1)以及[Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1)。

Llama-3.1-Nemotron-Ultra-253B-v1是基于[Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)（又称*参考模型*）衍生的大语言模型(Large Language Model, LLM)。
Llama-3.3-Nemotron-Super-49B-v1是基于[Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)（又称*参考模型*）衍生的大语言模型(LLM)。Llama-3.1-Nemotron-Nano-8B-v1是基于[Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)（又称*参考模型*）衍生的大语言模型(LLM)。上述模型均针对人类对话偏好与任务场景进行了对齐优化。

这些模型在模型精度与运行效率之间实现了出色的平衡。运行效率（吞吐量）直接对应计算成本的节约。通过采用创新的神经架构搜索(Neural Architecture Search, NAS)方法，我们大幅降低了模型的内存占用，支持更大规模的推理工作负载。该NAS方法可实现在精度-效率权衡曲线中选取期望的平衡点。所有模型均支持128K上下文长度。

本次数据集发布是模型开发与优化领域在开放性与透明度上的重要进展。除发布训练技术、工具与最终模型权重外，我们还公开了完整的训练数据集，NVIDIA此举旨在支持社区对本方案进行复现与进一步优化。

## 数据分布
| 类别 | 样本数量 |
|------------|--------------|
| 数学 | 22,066,397 |
| 代码 | 10,108,883 |
| 科学 | 708,920 |
| 指令遵循 | 56,339 |
| 对话 | 39,792 |
| 安全 | 31,426 |

## 数据筛选
用户可基于前述元数据架构下载数据集子集。以下为下载数学与代码数据的示例脚本：

from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])

## 提示词
提示词来源包括公开开放语料与合成生成两类。所有回复均由公开开放模型合成生成。
研究人员先对提示词进行提取，再基于质量与复杂度要求进行筛选，或直接生成符合质量与复杂度标准的提示词。筛选流程包括剔除不合规提示词、答案易于猜测的提示词，以及存在语法错误的提示词。

## 回复内容
回复由多款模型合成生成，部分提示词包含开启与关闭推理模式下的两种回复，用于训练模型区分两种推理模式。
本数据集构建过程中使用的模型如下：
| 模型名称 | 样本数量 |
|----------------------------------------------------------|--------------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## 许可/使用条款
本数据集的每条样本均标注了所属许可类型。数据集主体采用CC-BY-4.0许可，少量来自Wildchat的提示词采用ODC-BY许可，少量来自StackOverflow的提示词采用CC-BY-SA许可。
本数据集包含使用Llama-3.3-70B-Instruct、Llama-3.1-Nemotron-70B-Instruct以及Llama-3.3-Nemotron-70B-Feedback/Edit/Select（NVIDIA自研模型）生成的合成数据。若使用本数据集创建、训练、微调或以其他方式优化AI模型并进行分发或公开提供，则该AI模型需遵守Llama 3.1社区许可协议与Llama 3.3社区许可协议中的再分发与使用要求。

**数据开发者：** NVIDIA

### 使用场景：<br>
适用于训练AI智能体(AI Agent)系统、聊天机器人、检索增强生成(Retrieval-Augmented Generation, RAG)系统及其他AI赋能应用的开发者。<br>

### 发布日期：<br>
2025年4月8日<br>

## 数据版本
1.1（2025年4月8日）

## 预期用途
Llama Nemotron后训练数据集旨在面向社区，用于持续优化开源模型。用户可自由使用该数据集进行模型训练与评估。

## 伦理考量
NVIDIA认为可信AI是一项共同责任，我们已建立相关政策与实践规范，以支持各类AI应用的开发。开发者在按照服务条款下载或使用本数据集时，应与内部模型团队协作，确保该模型符合相关行业与应用场景的要求，并防范可能出现的产品误用。
请[在此处](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)提交安全漏洞或NVIDIA AI相关问题反馈。

## 数据自愿移除机制
NVIDIA已完成合规审查，确保数据中不包含机密信息、个人可识别信息(Personally Identifiable Information, PII)或受版权保护的材料。若您在审阅或使用本数据集时发现上述或其他相关问题，请联系[e-mail]。

## 引用

@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}