Llama-Nemotron-Post-Training-Dataset

# Llama-Nemotron-Post-Training-Dataset-v1.1 Release

**Update [4/8/2025]:**

**v1.1:** We are releasing an additional 2.2M Math and 500K Code Reasoning Data in support of our release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1). 🎉
## Data Overview

This dataset is a compilation of SFT and RL data that supports improvements of math, code, general reasoning, and instruction following capabilities of the original Llama instruct model, in support of NVIDIA’s release of [Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1), [Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1) and [Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1).

Llama-3.1-Nemotron-Ultra-253B-v1 is a large language model (LLM) which is a derivative of [Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) (AKA the *reference model*).
Llama-3.3-Nemotron-Super-49B-v1 is an LLM which is a derivative of [Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) (AKA the *reference model*). Llama-3.1-Nemotron-Nano-8B-v1 is an LLM which is a derivative of [Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) (AKA the *reference model*). They are aligned for human chat preferences, and tasks.

These models offer a great tradeoff between model accuracy and efficiency. Efficiency (throughput) directly translates to savings. Using a novel Neural Architecture Search (NAS) approach, we greatly reduce the model’s memory footprint and enable larger workloads. This NAS approach enables the selection of a desired point in the accuracy-efficiency tradeoff. The models support a context length of 128K.

This dataset release represents a significant move forward in openness and transparency in model development and improvement. By releasing the complete training set, in addition to the training technique, tools and final model weights, NVIDIA supports both the re-creation and the improvement of our approach.

## Data distribution

| Category | Value |
|----------|-----------|
| math | 22,066,397|
| code | 10,108,883 |
| science | 708,920 |
| instruction following | 56,339 |
| chat | 39,792 |
| safety | 31,426 |

## Filtering the data

Users can download subsets of the data based on the metadata schema described above. Example script for downloading code and math as follows:
```
from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])
```

## Prompts

Prompts have been sourced from either public and open corpus or synthetically generated. All responses have been synthetically generated from public and open models.

The prompts were extracted, and then filtered for quality and complexity, or generated to meet quality and complexity requirements. This included filtration such as removing inconsistent prompts, prompts with answers that are easy to guess, and removing prompts with incorrect syntax.

## Responses

Responses were synthetically generated by a variety of models, with some prompts containing responses for both reasoning on and off modes, to train the model to distinguish between two modes.

Models that were used in the creation of this dataset:

| Model | Number of Samples |
|----------|-----------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## License/Terms of Use

The dataset contains information about license type on a per sample basis. The dataset is predominantly CC-BY-4.0, with a small subset of prompts from Wildchat having an ODC-BY license and a small subset of prompts from StackOverflow with CC-BY-SA license.

This dataset contains synthetic data created using Llama-3.3-70B-Instruct, Llama-3.1-Nemotron-70B-Instruct and
Llama-3.3-Nemotron-70B-Feedback/Edit/Select (ITS models). If this dataset is used to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, such AI model may be subject to redistribution and use requirements in the Llama 3.1 Community License Agreement and Llama 3.3 Community License Agreement.

**Data Developer:** NVIDIA

### Use Case: <br>
Developers training AI Agent systems, chatbots, RAG systems, and other AI-powered applications. <br>

### Release Date: <br>
4/8/2025 <br>

## Data Version
1.1 (4/8/2025)

## Intended use

The Llama Nemotron Post-Training Dataset is intended to be used by the community to continue to improve open models. The data may be freely used to train and evaluate.

## Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Data Opt-Out:
NVIDIA has undertaken legal review to ensure there is no confidential, PII or copyright materials. If, when reviewing or using this dataset, you identify issues with the data itself, such as those listed above, please contact [e-mail].

## Citation
```
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}
```

# Llama-Nemotron 后训练数据集 v1.1 正式发布

**更新 [2025年4月8日]:**

**v1.1：** 为配合[Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)的正式发布，我们额外新增了220万条数学推理数据与50万条代码推理数据。🎉

## 数据集概览

本数据集为监督微调（Supervised Fine-Tuning, SFT）与强化学习（Reinforcement Learning, RL）数据的合集，旨在提升原生Llama指令模型在数学推理、代码生成、通用推理与指令遵循方面的能力，以配合NVIDIA发布[Llama-3.1-Nemotron-Ultra-253B-v1](https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)、[Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1)与[Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1)三款模型。

Llama-3.1-Nemotron-Ultra-253B-v1为大语言模型（Large Language Model, LLM），是[Meta Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct，又称*参考模型*)的衍生模型。Llama-3.3-Nemotron-Super-49B-v1为大语言模型，是[Meta Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct，又称*参考模型*)的衍生模型。Llama-3.1-Nemotron-Nano-8B-v1为大语言模型，是[Meta Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct，又称*参考模型*)的衍生模型。上述模型均针对人类对话偏好与任务需求完成对齐。

这些模型在模型精度与运行效率间实现了出色的平衡。运行效率（吞吐量）直接对应计算成本的节约。我们通过创新性的神经架构搜索（Neural Architecture Search, NAS）方法，大幅降低了模型的内存占用，支持更大规模的推理任务。该NAS方法可灵活选择精度与效率间的最优平衡点。所有模型均支持128K上下文长度。

本次数据集发布是模型开发与迭代过程中，在开放性与透明度层面的重要突破。NVIDIA不仅发布了完整训练集，还同步公开了训练技术、配套工具与最终模型权重，助力社区复现并进一步优化我们的研发方案。

## 数据分布

| 数据类别 | 样本数量 |
|----------|---------|
| 数学推理 | 22,066,397 |
| 代码生成 | 10,108,883 |
| 科学推理 | 708,920 |
| 指令遵循 | 56,339 |
| 对话交互 | 39,792 |
| 安全对齐 | 31,426 |

## 数据筛选

用户可基于上述元数据模式下载对应子集的数据。以下为下载数学与代码数据的示例脚本：

from datasets import load_dataset
ds = load_dataset("nvidia/Llama-Nemotron-Post-Training-Dataset", "SFT", split=["code", "math"])

## 提示词（Prompt）

本数据集的提示词均来自公开开放语料或通过合成方式生成，所有回复均由公开开放模型合成得到。

我们先提取原始提示词，随后基于质量与复杂度要求进行筛选，或直接生成符合标准的提示词。筛选流程包括移除格式不一致、答案易于猜测以及存在语法错误的提示词。

## 回复内容

回复内容由多款模型合成生成，部分提示词包含两种推理模式（开启/关闭）下的回复，用于训练模型区分两类推理模式。

用于构建本数据集的模型如下：

| 模型名称 | 样本数量 |
|----------|---------|
| Llama-3.3-70B-Instruct | 420,021 |
| Llama-3.1-Nemotron-70B-Instruct | 31,218 |
| Llama-3.3-Nemotron-70B-Feedback/Edit/Select | 22,644 |
| Mixtral-8x22B-Instruct-v0.1 | 31,426 |
| DeepSeek-R1 | 3,934,627 |
| Qwen-2.5-Math-7B-Instruct | 19,840,970 |
| Qwen-2.5-Coder-32B-Instruct | 8,917,167 |
| Qwen-2.5-72B-Instruct | 464,658 |
| Qwen-2.5-32B-Instruct | 2,297,175 |

## 许可证与使用条款

本数据集的每条样本均标注了对应的许可证类型。数据集主体采用CC-BY-4.0许可证，少量来自Wildchat的提示词采用ODC-BY许可证，另有少量来自StackOverflow的提示词采用CC-BY-SA许可证。

本数据集包含通过Llama-3.3-70B-Instruct、Llama-3.1-Nemotron-70B-Instruct与Llama-3.3-Nemotron-70B-Feedback/Edit/Select（ITS模型）合成生成的数据。若使用本数据集创建、训练、微调或优化AI模型并进行分发或公开部署，则该AI模型需遵守Llama 3.1社区许可协议与Llama 3.3社区许可协议中的重新分发与使用要求。

**数据集开发方：** NVIDIA

### 适用场景：<br>
用于训练AI智能体（AI Agent）系统、聊天机器人、检索增强生成（Retrieval-Augmented Generation, RAG）系统及其他AI驱动应用的开发者。<br>

### 发布日期：<br>
2025年4月8日<br>

## 数据集版本
v1.1（2025年4月8日）

## 使用意图

本数据集旨在供社区进一步优化开源模型，可免费用于模型训练与评估工作。

## 伦理考量

NVIDIA坚信可信AI是全社会的共同责任，我们已建立相关政策与实践规范，以支持各类AI应用的研发。开发者在按照服务条款下载或使用本数据集时，应与内部模型团队协作，确保该模型符合相关行业与应用场景的要求，并防范潜在的产品滥用风险。

请通过[此链接](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)提交安全漏洞或NVIDIA AI相关问题反馈。

## 数据合规退出机制

NVIDIA已完成法律合规审查，确保本数据集不包含机密信息、个人可识别信息（Personally Identifiable Information, PII）或侵权内容。若您在审阅或使用本数据集时发现上述或其他相关问题，请联系[e-mail]。

## 引用格式

@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}

搜集汇总

背景与挑战

背景概述

Llama-Nemotron-Post-Training-Dataset是一个用于提升Llama-3.1-Nemotron系列模型在数学、代码、推理和指令遵循方面能力的训练数据集，包含多种类别的合成数据，遵循Apache License 2.0许可，旨在支持开放模型的持续改进。