leideng/Dolci-Think-SFT-7B-4K-Plus

---
dataset_info:
features:
- name: messages
list:
- name: content
dtype: string
- name: role
dtype: string
- name: dataset_source
dtype: string
- name: id
dtype: string
splits:
- name: train
num_bytes: 77877465405
num_examples: 2268178
download_size: 36139346939
dataset_size: 77877465405
configs:
- config_name: 4k_plus
data_files:
- split: train
path: data_4k/train-*
default: true
- config_name: full
data_files:
- split: train
path: data/train-*
---

# Note

>[!NOTE]
>
>This is the filtered version of [allenai/Dolci-Think-SFT-7B](https://huggingface.co/datasets/allenai/Dolci-Think-SFT-7B) where only thoese data samples with more than 4096 tokens by Qwen3 tokenizer are kept.

# Dolci-Think-SFT

Sources include a mixture of existing reasoning traces:

* [OpenThoughts 3](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M) (Apache 2.0): Extended to 32K context length and downsampled code prompts to 16X multiple, to 941,166 total prompts. Access our version, Dolci OpenThoughts 3 here.
* [SYNTHETIC-2](https://huggingface.co/datasets/PrimeIntellect/SYNTHETIC-2-SFT-verified) (Apache 2.0) via the SFT-Verified split, 104,569 prompts.
* [Nemotron Post-training dataset](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v1) (CC BY 4), code split only, 113,777 prompts.

New prompts and new reasoning traces from us (all ODC-BY-1.0):

* Dolci Think Persona IF: New precise instruction following prompts and traces created with [Nvidia's Nemotron Post-training Personas](https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA). 223,123 prompts.
* Dolci Precise IF: New multi-constraint instruction following data building off Pyatkin, Valentina, et al. "[Generalizing Verifiable Instruction Following](https://arxiv.org/abs/2507.02833)." (2025). 135,792 prompts.
* [Dolci Think Python](https://huggingface.co/datasets/allenai/Dolci-Think-SFT-Python): 466,677 prompts (subsampled from larger mix).

Existing prompts with new reasoning traces, largely repurposed from Tülu 3 / OLMo 2, with new traces generated by a mix of DeepSeek R1 and DeepSeek R1 0528:

* [WildChat](https://huggingface.co/datasets/allenai/WildChat-1M) (ODC-BY-1.0), 83,054 prompts.
* [OpenAssistant Guanaco](https://huggingface.co/datasets/OpenAssistant/oasst1) (Apache 2.0), 6,800 prompts.
* [CoCoNot](https://huggingface.co/datasets/allenai/coconot) (ODC-BY-1.0), 10,227 prompts.
* [WildGuardMix ](https://huggingface.co/datasets/allenai/wildguardmix) (Apache 2.0), 38,315 prompts.
* [WildJailbreak](https://huggingface.co/datasets/allenai/wildjailbreak) (ODC-BY-1.0) 41,100 prompts.
* [Aya](https://huggingface.co/datasets/CohereForAI/aya_dataset) (Apache 2.0), 98,597 prompts.
* [TableGPT](https://huggingface.co/datasets/LipengCS/Table-GPT) (MIT), 4,981 prompts.
* Olmo Identity Prompts, 58 samples (we trained with 290, 5 repetitions per prompt, uploaded single repetition to HuggingFace)

The counts are smaller than the original prompt sources pulled from Tülu 3 / OLMo 2 due to more extensive filtering for data quality and by topics within the Azure API (blocked requests).
This dataset was used for 7B post-training, the [7B dataset](https://huggingface.co/datasets/allenai/Dolci-Think-SFT) is slightly different.

## Dataset Structure
Each example in the dataset contains the standard instruction-tuning data points as follow:

- `id` (str): a unique identifier
- `messages` (list): message format used for supervised fine-tuning (this contains user prompt and assistant responses)
- `source` (str): the source dataset for the given sample

Every datapoint contains the model's reasoning in `<think>...</think>` and NO `<answer>...</answer>` tags -- the answer follows directly after `</think>`.

## Model Family

| **Stage** | **Olmo 3 7B Think** | **Olmo 3 32B Think** | **Olmo 3 7B Instruct** |
|--------------------------|-----------------------|------------------------|---------------------------|
| **Base Model** | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) | [Olmo-3-32B](https://huggingface.co/allenai/Olmo-3-1125-32B) | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) |
| **SFT** | [Olmo-3-7B-Think-SFT](https://huggingface.co/allenai/Olmo-3-7B-Think-SFT) | [Olmo-3-32B-Think-SFT](https://huggingface.co/allenai/Olmo-3-32B-Think-SFT) | [Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) |
| **DPO** | [Olmo-3-7B-Think-DPO](https://huggingface.co/allenai/Olmo-3-7B-Think-DPO) | [Olmo-3-32B-Think-DPO](https://huggingface.co/allenai/Olmo-3-32B-Think-DPO) | [Olmo-3-7B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3-7B-Instruct-DPO) |
| **Final Models (RLVR)** | [Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think) | [Olmo-3-32B-Think](https://huggingface.co/allenai/Olmo-3-32B-Think) | [Olmo-3-7B-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) |

## License

This dataset is licensed under ODC-BY. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).

## Citation

```
@misc{olmo2025olmo3,
title={Olmo 3},
author={Team Olmo and Allyson Ettinger and Amanda Bertsch and Bailey Kuehl and David Graham and David Heineman and Dirk Groeneveld and Faeze Brahman and Finbarr Timbers and Hamish Ivison and Jacob Morrison and Jake Poznanski and Kyle Lo and Luca Soldaini and Matt Jordan and Mayee Chen and Michael Noukhovitch and Nathan Lambert and Pete Walsh and Pradeep Dasigi and Robert Berry and Saumya Malik and Saurabh Shah and Scott Geng and Shane Arora and Shashank Gupta and Taira Anderson and Teng Xiao and Tyler Murray and Tyler Romero and Victoria Graf and Akari Asai and Akshita Bhagia and Alexander Wettig and Alisa Liu and Aman Rangapur and Chloe Anastasiades and Costa Huang and Dustin Schwenk and Harsh Trivedi and Ian Magnusson and Jaron Lochner and Jiacheng Liu and Lester James V. Miranda and Maarten Sap and Malia Morgan and Michael Schmitz and Michal Guerquin and Michael Wilson and Regan Huff and Ronan Le Bras and Rui Xin and Rulin Shao and Sam Skjonsberg and Shannon Zejiang Shen and Shuyue Stella Li and Tucker Wilde and Valentina Pyatkin and Will Merrill and Yapei Chang and Yuling Gu and Zhiyuan Zeng and Ashish Sabharwal and Luke Zettlemoyer and Pang Wei Koh and Ali Farhadi and Noah A. Smith and Hannaneh Hajishirzi},
year={2025},
eprint={2512.13961},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2512.13961},
}
```

数据集信息：
特征：
- 字段名：messages
列表类型：
- 字段名：content
数据类型：字符串
- 字段名：role
数据类型：字符串
- 字段名：dataset_source
数据类型：字符串
- 字段名：id
数据类型：字符串
划分集：
- 划分名：train
字节数：77877465405
样本数：2268178
下载大小：36139346939
数据集总大小：77877465405
配置项：
- 配置名：4k_plus
数据文件：
- 划分：train
路径：data_4k/train-*
为默认配置
- 配置名：full
数据文件：
- 划分：train
路径：data/train-*

# 注意

>[!NOTE]
本数据集是[allenai/Dolci-Think-SFT-7B](https://huggingface.co/datasets/allenai/Dolci-Think-SFT-7B)的过滤版本，仅保留了经通义千问3分词器（Qwen3 tokenizer）统计Token数超过4096的样本。

# Dolci-Think-SFT

数据源包含多种现有推理轨迹：
* [OpenThoughts 3](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M)（遵循Apache 2.0协议）：扩展至32K上下文长度，将代码提示按16倍下采样，最终得到941166条提示。可在此获取我们的OpenThoughts 3衍生版本：Dolci OpenThoughts 3。
* [SYNTHETIC-2](https://huggingface.co/datasets/PrimeIntellect/SYNTHETIC-2-SFT-verified)（遵循Apache 2.0协议）：采用SFT-Verified划分，包含104569条提示。
* [Nemotron Post-training dataset](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v1)（遵循CC BY 4协议）：仅使用代码划分，包含113777条提示。

我们自研的全新提示与推理轨迹（均遵循ODC-BY-1.0协议）：
* Dolci Think Persona IF：基于[Nvidia's Nemotron Post-training Personas](https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA)构建的高精度指令遵循提示与推理轨迹，包含223123条提示。
* Dolci Precise IF：基于Pyatkin, Valentina等人2025年发表的论文《Generalizing Verifiable Instruction Following》（arXiv:2507.02833）构建的多约束指令遵循数据集，包含135792条提示。
* [Dolci Think Python](https://huggingface.co/datasets/allenai/Dolci-Think-SFT-Python)：466677条提示（从更大规模混合数据中采样得到）。

基于现有提示生成全新推理轨迹的数据集，主要源自Tülu 3与OLMo 2，推理轨迹由DeepSeek R1及DeepSeek R1 0528混合生成：
* [WildChat](https://huggingface.co/datasets/allenai/WildChat-1M)（遵循ODC-BY-1.0协议）：83054条提示。
* [OpenAssistant Guanaco](https://huggingface.co/datasets/OpenAssistant/oasst1)（遵循Apache 2.0协议）：6800条提示。
* [CoCoNot](https://huggingface.co/datasets/allenai/coconot)（遵循ODC-BY-1.0协议）：10227条提示。
* [WildGuardMix](https://huggingface.co/datasets/allenai/wildguardmix)（遵循Apache 2.0协议）：38315条提示。
* [WildJailbreak](https://huggingface.co/datasets/allenai/wildjailbreak)（遵循ODC-BY-1.0协议）：41100条提示。
* [Aya](https://huggingface.co/datasets/CohereForAI/aya_dataset)（遵循Apache 2.0协议）：98597条提示。
* [TableGPT](https://huggingface.co/datasets/LipengCS/Table-GPT)（遵循MIT协议）：4981条提示。
* Olmo Identity Prompts：58条样本（我们训练时使用了290条，每条提示重复5次，本次上传仅保留单份重复版本至HuggingFace平台）。

由于针对数据质量与Azure API内的主题进行了更严格的过滤（含被拦截请求），本数据集的样本数少于最初从Tülu 3与OLMo 2中提取的原始提示源规模。本数据集用于7B模型的后训练，[7B版本数据集](https://huggingface.co/datasets/allenai/Dolci-Think-SFT)存在细微差异。

## 数据集结构

本数据集的每条样本均包含标准的监督微调（Supervised Fine-Tuning, SFT）数据格式，具体如下：
- `id`（字符串类型）：唯一标识符
- `messages`（列表类型）：用于监督微调的消息格式（包含用户提示与助手回复）
- `source`（字符串类型）：当前样本所属的源数据集

每条数据均包含模型的推理过程，包裹在`<think>...</think>`标签内，且不包含`<answer>...</answer>`标签——答案直接紧跟在`</think>`之后。

## 模型家族

| **阶段** | **Olmo 3 7B Think** | **Olmo 3 32B Think** | **Olmo 3 7B Instruct** |
|--------------------------|-----------------------|------------------------|---------------------------|
| **基础模型** | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) | [Olmo-3-32B](https://huggingface.co/allenai/Olmo-3-1125-32B) | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) |
| **监督微调（SFT）** | [Olmo-3-7B-Think-SFT](https://huggingface.co/allenai/Olmo-3-7B-Think-SFT) | [Olmo-3-32B-Think-SFT](https://huggingface.co/allenai/Olmo-3-32B-Think-SFT) | [Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) |
| **直接偏好优化（DPO）** | [Olmo-3-7B-Think-DPO](https://huggingface.co/allenai/Olmo-3-7B-Think-DPO) | [Olmo-3-32B-Think-DPO](https://huggingface.co/allenai/Olmo-3-32B-Think-DPO) | [Olmo-3-7B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3-7B-Instruct-DPO) |
| **最终模型（RLVR）** | [Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think) | [Olmo-3-32B-Think](https://huggingface.co/allenai/Olmo-3-32B-Think) | [Olmo-3-7B-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) |

## 授权协议

本数据集遵循ODC-BY协议授权，仅可用于研究与教育用途，并需符合Ai2的[负责任使用指南](https://allenai.org/responsible-use)。

## 引用

@misc{olmo2025olmo3,
title={Olmo 3},
author={Team Olmo and Allyson Ettinger and Amanda Bertsch and Bailey Kuehl and David Graham and David Heineman and Dirk Groeneveld and Faeze Brahman and Finbarr Timbers and Hamish Ivison and Jacob Morrison and Jake Poznanski and Kyle Lo and Luca Soldaini and Matt Jordan and Mayee Chen and Michael Noukhovitch and Nathan Lambert and Pete Walsh and Pradeep Dasigi and Robert Berry and Saumya Malik and Saurabh Shah and Scott Geng and Shane Arora and Shashank Gupta and Taira Anderson and Teng Xiao and Tyler Murray and Tyler Romero and Victoria Graf and Akari Asai and Akshita Bhagia and Alexander Wettig and Alisa Liu and Aman Rangapur and Chloe Anastasiades and Costa Huang and Dustin Schwenk and Harsh Trivedi and Ian Magnusson and Jaron Lochner and Jiacheng Liu and Lester James V. Miranda and Maarten Sap and Malia Morgan and Michael Schmitz and Michal Guerquin and Michael Wilson and Regan Huff and Ronan Le Bras and Rui Xin and Rulin Shao and Sam Skjonsberg and Shannon Zejiang Shen and Shuyue Stella Li and Tucker Wilde and Valentina Pyatkin and Will Merrill and Yapei Chang and Yuling Gu and Zhiyuan Zeng and Ashish Sabharwal and Luke Zettlemoyer and Pang Wei Koh and Ali Farhadi and Noah A. Smith and Hannaneh Hajishirzi},
year={2025},
eprint={2512.13961},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2512.13961},
}