Denzounion/Llama-Nemotron-Post-Training-Dataset

---
license: cc-by-4.0
configs:
- config_name: SFT
data_files:
- split: code
path: SFT/code/*.jsonl
- split: math
path: SFT/math/*.jsonl
- split: science
path: SFT/science/*.jsonl
- split: chat
path: SFT/chat/*.jsonl
- split: safety
path: SFT/safety/*.jsonl
default: true
- config_name: RL
data_files:
- split: instruction_following
path: RL/instruction_following/*.jsonl
---
# Llama-Nemotron-Post-Training-Dataset-v1.1 Release

**Update [4/8/2025]:**

**v1.1:** We are releasing an additional 2.2M Math and 500K Code Reasoning Data in support of our release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1). 🎉
## Data Overview

This dataset is a compilation of SFT and RL data that supports improvements of math, code, general reasoning, and instruction following capabilities of the original Llama instruct model, in support of NVIDIA’s release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1), [Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1) and [Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1).

Llama-3.1-Nemotron-Ultra-253B-v1 is a large language model (LLM) which is a derivative of [Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) (AKA the *reference model*).
Llama-3.3-Nemotron-Super-49B-v1 is an LLM which is a derivative of [Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) (AKA the *reference model*). Llama-3.1-Nemotron-Nano-8B-v1 is an LLM which is a derivative of [Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) (AKA the *reference model*). They are aligned for human chat preferences, and tasks.

These models offer a great tradeoff between model accuracy and efficiency. Efficiency (throughput) directly translates to savings. Using a novel Neural Architecture Search (NAS) approach, we greatly reduce the model’s memory footprint and enable larger workloads. This NAS approach enables the selection of a desired point in the accuracy-efficiency tradeoff. The models support a context length of 128K.

This dataset release represents a significant move forward in openness and transparency in model development and improvement. By releasing the complete training set, in addition to the training technique, tools and final model weights, NVIDIA supports both the re-creation and the improvement of our approach.

## Data distribution

| Category | Value |
|----------|-----------|
| math | 22,066,397|
| code | 10,108,883 |
| science | 708,920 |
| instruction following | 56,339 |
| chat | 39,792 |
| safety | 31,426 |

## Filtering the data

Users can download subsets of the data based on the metadata schema described above. Example script for downloading code and math as follows:
```
from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])
```

## Prompts

Prompts have been sourced from either public and open corpus or synthetically generated. All responses have been synthetically generated from public and open models.

The prompts were extracted, and then filtered for quality and complexity, or generated to meet quality and complexity requirements. This included filtration such as removing inconsistent prompts, prompts with answers that are easy to guess, and removing prompts with incorrect syntax.

## Responses

Responses were synthetically generated by a variety of models, with some prompts containing responses for both reasoning on and off modes, to train the model to distinguish between two modes.

Models that were used in the creation of this dataset:

| Model | Number of Samples |
|----------|-----------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## License/Terms of Use

The dataset contains information about license type on a per sample basis. The dataset is predominantly CC-BY-4.0, with a small subset of prompts from Wildchat having an ODC-BY license and a small subset of prompts from StackOverflow with CC-BY-SA license.

This dataset contains synthetic data created using Llama-3.3-70B-Instruct, Llama-3.1-Nemotron-70B-Instruct and
Llama-3.3-Nemotron-70B-Feedback/Edit/Select (ITS models). If this dataset is used to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, such AI model may be subject to redistribution and use requirements in the Llama 3.1 Community License Agreement and Llama 3.3 Community License Agreement.

**Data Developer:** NVIDIA

### Use Case: <br>
Developers training AI Agent systems, chatbots, RAG systems, and other AI-powered applications. <br>

### Release Date: <br>
4/8/2025 <br>

## Data Version
1.1 (4/8/2025)

## Intended use

The Llama Nemotron Post-Training Dataset is intended to be used by the community to continue to improve open models. The data may be freely used to train and evaluate.

## Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Data Opt-Out:
NVIDIA has undertaken legal review to ensure there is no confidential, PII or copyright materials. If, when reviewing or using this dataset, you identify issues with the data itself, such as those listed above, please contact [e-mail].

## Citation
```
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}
```

### 数据集配置
许可证：CC-BY-4.0
配置项：
- 配置名称：SFT
数据文件：
- 划分集：code（代码），路径：SFT/code/*.jsonl
- 划分集：math（数学），路径：SFT/math/*.jsonl
- 划分集：science（科学），路径：SFT/science/*.jsonl
- 划分集：chat（聊天），路径：SFT/chat/*.jsonl
- 划分集：safety（安全），路径：SFT/safety/*.jsonl
默认启用：是
- 配置名称：RL
数据文件：
- 划分集：instruction_following（指令遵循），路径：RL/instruction_following/*.jsonl

# Llama-Nemotron-Post-Training-Dataset-v1.1 发布版

**更新 [2025年4月8日]：**

**v1.1版本：** 我们新增发布了220万条数学数据与50万条代码推理数据，以配合[NVIDIA Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)的发布。🎉

## 数据概览
本数据集为监督微调（Supervised Fine-Tuning, SFT）与强化学习（Reinforcement Learning, RL）数据的合集，用于提升原生Llama指令模型的数学、代码、通用推理与指令遵循能力，以配合NVIDIA发布的[Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)、[Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1)以及[Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1)。

Llama-3.1-Nemotron-Ultra-253B-v1是一款大语言模型（Large Language Model，简称LLM），衍生自[Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)（下称*参考模型*）。
Llama-3.3-Nemotron-Super-49B-v1同样为大语言模型，衍生自[Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)（下称*参考模型*）。Llama-3.1-Nemotron-Nano-8B-v1亦是一款大语言模型，衍生自[Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)（下称*参考模型*）。上述模型均针对人类聊天偏好与任务需求完成对齐。

上述模型在模型精度与效率之间实现了极佳的平衡。效率（吞吐量）直接对应计算成本的节约。通过采用新颖的神经架构搜索（Neural Architecture Search, NAS）方法，我们大幅降低了模型的内存占用，支持更大规模的计算任务。该神经架构搜索方法可让用户在精度与效率的权衡区间中选择符合需求的平衡点。上述模型均支持128K的上下文长度。

本次数据集的发布是模型开发与优化领域在开放性与透明度方面的重要进步。NVIDIA除发布训练技术、工具与最终模型权重外，还同步发布完整训练集，以支持社区复刻并优化本项目的技术方案。

## 数据分布
| 类别 | 样本量 |
|----------|-----------|
| 数学（math） | 22,066,397 |
| 代码（code） | 10,108,883 |
| 科学（science） | 708,920 |
| 指令遵循（instruction_following） | 56,339 |
| 聊天（chat） | 39,792 |
| 安全（safety） | 31,426 |

## 数据筛选
用户可基于上述元数据架构下载数据子集。以下为下载代码与数学数据的示例脚本：
python
from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])

## 提示词
提示词均来自公开开放语料或人工合成。所有回复均由公开开放模型合成生成。

提示词经提取后，会基于质量与复杂度要求进行筛选，或直接按该要求生成。筛选流程包括移除不一致的提示词、答案易被猜测的提示词，以及语法错误的提示词。

## 回复内容
回复内容由多种模型合成生成，部分提示词包含开启与关闭推理模式的回复，用于训练模型区分两种推理模式。

用于构建本数据集的模型如下：
| 模型 | 样本数量 |
|----------|-----------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## 许可证与使用条款
本数据集的每条样本均标注了对应的许可证类型。数据集主体采用CC-BY-4.0许可证，少量来自Wildchat的提示词采用ODC-BY许可证，另有少量来自StackOverflow的提示词采用CC-BY-SA许可证。

本数据集包含使用Llama-3.3-70B-Instruct、Llama-3.1-Nemotron-70B-Instruct以及Llama-3.3-Nemotron-70B-Feedback/Edit/Select（ITS模型）生成的合成数据。若使用本数据集创建、训练、微调或以其他方式优化并分发或公开提供AI模型，则该AI模型需遵守Llama 3.1社区许可协议与Llama 3.3社区许可协议中的再分发与使用要求。

**数据开发者：** NVIDIA

### 应用场景：
适用于训练AI智能体（AI Agent）系统、聊天机器人、检索增强生成（Retrieval-Augmented Generation, RAG）系统及其他AI驱动应用的开发者。

### 发布日期：
2025年4月8日

## 数据集版本
1.1（2025年4月8日）

## 预期用途
Llama Nemotron 后训练数据集旨在供社区持续优化开源模型，可免费用于模型训练与评估。

## 伦理考量
NVIDIA认为，可信人工智能是一项共同责任，我们已建立相关政策与实践规范，以支持各类AI应用的开发。开发者在按照服务条款下载或使用本数据集时，应与内部模型团队协作，确保模型符合相关行业与应用场景的要求，并防范可能出现的产品滥用问题。

请通过[此链接](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)提交安全漏洞或NVIDIA AI相关问题反馈。

## 数据移除机制
NVIDIA已完成法律审查，确保本数据集不包含机密信息、个人可识别信息（Personally Identifiable Information, PII）或侵权内容。若您在审阅或使用本数据集时发现上述或其他数据问题，请联系[e-mail]。

## 引用格式
bibtex
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}