fineinstructions/fineinstructions_nemotron

---
language:
- en
---

[](https://huggingface.co/fineinstructions)

**✨ Note:** For all FineInstructions resources please visit: https://huggingface.co/fineinstructions

----

This dataset is ~1B+ synthetic instruction-answer pairs or ~300B tokens created using the [FineInstructions pipeline](https://huggingface.co/fineinstructions).

The FineInstructions pipeline was run over the raw pre-training documents in the Nemotron-CC pre-training corpus (a subset of high-quality documents from CommonCrawl). See our paper for more details.

Each `.parquet` file in the [`data` folder](https://huggingface.co/datasets/fineinstructions/fineinstructions_nemotron/tree/main/data) has a corresponding `judge-*.json` file that contains an automatic judgement score of the quality of the synthetic instruction-answer pair on a Likert score (1-5) where 5 is the highest-quality.

<!-- Autocitation -->

--------------------

If you use this project in your research please cite:
```
@article{patel2026fineinstructions,
title={FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale},
author={Patel, Ajay and Raffel, Colin and Callison-Burch, Chris},
journal={arXiv preprint arXiv:2601.22146},
year={2026},
archivePrefix={arXiv},
primaryClass={cs.CL},
doi={10.48550/arXiv.2601.22146}
}
```