Mixture-of-Thoughts

<img src="mot-thumbnail.png" alt="Centered Image" style="display: block; margin: 0 auto;" width="500">

# Dataset summary

Mixture-of-Thoughts is a curated dataset of 350k verified reasoning traces distilled from [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1). The dataset spans tasks in mathematics, coding, and science, and is designed to teach language models to reason step-by-step. It was used in the Open R1 project to train [OpenR1-Distill-7B](https://huggingface.co/open-r1/OpenR1-Distill-7B), an SFT model that replicates the reasoning capabilities of [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) from the same base model.

To load the dataset, run:

```python
from datasets import load_dataset

dataset = load_dataset("open-r1/Mixture-of-Thoughts", "all", split="train")

# Load a specific domain
dataset_math = load_dataset("open-r1/Mixture-of-Thoughts", "math", split="train")
```

## Dataset composition

Mixture-of-Thoughts is composed of three domains: math, code, and science. Each domain contains reasoning traces that are designed to teach language models to reason step-by-step. The dataset is structured as follows:

- **math**: 93.7k reasoning traces for mathematical problems, sourced from the `default` subset of [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k)
- **code**: 83.1k reasoning traces for competitive programming problems in Python and C++, sourced from the `solutions` and `solutions_w_editorials` subsets of [open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots)
- **science**: 173k reasoning traces for scientific problems, sourced from the `science` subset of [nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset)
- **all**: Contains all reasoning traces from the three domains, for a total of 350k traces.

## Curation methodology

To optimise the data mixture, we followed the same methodology described in the [Phi-4-reasoning tech report](https://huggingface.co/papers/2504.21318), namely that mixtures can be optimised independently per domain, and then combined into a single dataset. For each ablation, we evaluate on AIME 2024, GPQA Diamond, and LiveCodeBench v4 every epoch and take the best performing model checkpoint. The figure below shows the results from post-training [open-r1/Qwen2.5-Math-7B-RoPE-300k](https://huggingface.co/open-r1/Qwen2.5-Math-7B-RoPE-300k) on each individual domain compared to the final mixture:

<img src="data_mix.png" alt="Centered Image" style="display: block; margin: 0 auto;">

Overall, we find that training on all domains simultaneously yields the best results. See the subsections below for more details on optimising the data mixture per domain.

> [!NOTE]
> We use LiveCodeBench v4 to accelerate evaluation during our ablations as it contains around half the problems of v5, yet is still representative of the full benchmark.

### Code

During the development of [open-r1/OlympicCoder-7B](https://huggingface.co/open-r1/OlympicCoder-7B), we observed that generating R1 reasoning traces in C++ produced better results on the challenging [IOI 2024 benchmark](https://github.com/huggingface/ioi), while Python traces produced better results on LiveCodeBench (a Python-only benchmark). To optimise the data mixture, we therefore used a mix of C++ and Python traces sourced from the following subsets of [open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots):

- `solutions`: we prompt R1 to solve the problem and produce code in C++.
- `solutions_py`: same as `solutions`, but with R1 prompted to produce code in Python.
- `solutions_w_editorials`: we prompt R1 to solve the problem and produce code, but also provide it with a human-written solution.
- `solutions_w_editorials_py`: same as `solutions_w_editorials`, but with R1 prompted to produce code in Python.

The figure below shows the evolution of our ablations on these subsets, using [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) as the base model:

<img src="code_mix.png" alt="Centered Image" style="display: block; margin: 0 auto;">

The individual experiments correspond to the following:

* **exp1 - exp3:** scaling the learning rate on the `solutions` subset from 1e-5 to 2e-5, and 4e-5 respectively.
* **exp4 - exp5:** measuring the impact of training on the `solutions_w_editorials` subset vs the combined `solutions` and `solutions_w_editorials` subsets.
* **exp6 - exp9:** measuring the impact of blending in Python traces from the `solutions_py` and `solutions_w_editorials_py` subsets. exp6 combines the `solutions_w_editorials` and `solutions_w_editorials_py` subsets, while exp7 combines the `solutions` and `solutions_py` subsets. Finally, exp8 combines all four subsets.

We found that combining all subsets of C++ and Python traces yielded the best results on LiveCodeBench. We also found that using this data mixture to fine-tune [open-r1/Qwen2.5-Coder-7B-RoPE-300k](https://huggingface.co/open-r1/Qwen2.5-Coder-7B-RoPE-300k) led to comparable performance improvements, which shows the effectiveness of our curation strategy.

### Math

For the math domain, we mostly focused on comparing the `default` and `extended` subsets of [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k). The `default` subset contains 93.7k reasoning traces, while the `extended` subset contains an additional 131k traces, containing simpler problems than the `default` subset. The figure below shows performance on each subset, using [Qwen/Qwen2.5-Math-7B-RoPE-300k](https://huggingface.co/Qwen/Qwen2.5-Math-7B-RoPE-300k) as the base model:

<img src="math_mix.png" alt="Centered Image" style="display: block; margin: 0 auto;">

Overall, we found that training on the `default` subset yielded better results than training on the `extended` subset, and that training on both subsets together yielded the best results. Nevertheless, we opted to use the `default` subset only for the final mixture, as including both would have led to a significant increase in the size of the dataset, for a modest improvement in performance.

### Science

For the science domain, we used the `science` subset of [nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset/viewer/SFT/science), which contains 483k reasoning traces. However, we found that the subset was too large to be used in its entirety, as it would have led to a significant increase in the size of the dataset. Instead, we selected the subset of traces where no Qwen models were used for prompt pre-processing--see this [discussion](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset/discussions/6) for more details. The result was 173k reasoning traces, which we used in the final mixture after ablating on the learning rate.

## Citation

If you find this dataset is useful in your own work, please consider citing it as follows, together with the source of the specific domain you are using:

```bibtex
@misc{openr1,
title = {Open R1: A fully open reproduction of DeepSeek-R1},
url = {https://github.com/huggingface/open-r1},
author = {Hugging Face},
month = {January},
year = {2025}
}
```

**open-r1/codeforces-cots**

```bibtex
@misc{penedo2025codeforces,
title={CodeForces CoTs},
author={Guilherme Penedo and Anton Lozhkov and Hynek Kydlíček and Loubna Ben Allal and Edward Beeching and Agustín Piqueres Lajarín and Quentin Gallouédec and Nathan Habib and Lewis Tunstall and Leandro von Werra},
year={2025},
publisher = {Hugging Face},
journal = {Hugging Face repository},
howpublished = {\url{https://huggingface.co/datasets/open-r1/codeforces-cots}}
}
```

**open-r1/OpenR1-Math-220k**

```bibtex
@misc{lozhkov2025openr1math220k,
title={OpenR1-Math-220k},
author={Anton Lozhkov and Hynek Kydlíček and Loubna Ben Allal and Guilherme Penedo and Edward Beeching and Quentin Gallouédec and Nathan Habib and Lewis Tunstall and Leandro von Werra},
year={2025},
publisher = {Hugging Face},
journal = {Hugging Face repository},
howpublished = {\url{https://huggingface.co/datasets/open-r1/OpenR1-Math-220k}}
}
```

**nvidia/Llama-Nemotron-Post-Training-Dataset**

```bibtex
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}
```

<img src="mot-thumbnail.png" alt="居中图片" style="display: block; margin: 0 auto;" width="500">

# 数据集概述

**混合思维（Mixture-of-Thoughts）** 是一个从[DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)中蒸馏得到的、包含35万条经过验证的推理轨迹的精选数据集。该数据集涵盖数学、编码与科学领域的任务，旨在赋能大语言模型（Large Language Model，LLM）进行逐步推理。本数据集被应用于Open R1项目，用于训练[OpenR1-Distill-7B](https://huggingface.co/open-r1/OpenR1-Distill-7B)——这是一个监督微调（Supervised Fine-Tuning, SFT）模型，可从同一基座模型复现[deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)的推理能力。

如需加载该数据集，请运行以下代码：

python
from datasets import load_dataset

dataset = load_dataset("open-r1/Mixture-of-Thoughts", "all", split="train")

# Load a specific domain
dataset_math = load_dataset("open-r1/Mixture-of-Thoughts", "math", split="train")

## 数据集构成

混合思维数据集包含三大领域：数学、编码与科学，每个领域均包含旨在教会大语言模型逐步推理的推理轨迹。数据集结构如下：

- **数学**：9.37万条针对数学问题的推理轨迹，源自[open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k)的`default`子集
- **编码**：8.31万条针对Python与C++竞赛编程问题的推理轨迹，源自[open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots)的`solutions`与`solutions_w_editorials`子集
- **科学**：17.3万条针对科学问题的推理轨迹，源自[nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset)的`science`子集
- **全量**：包含三大领域的全部推理轨迹，总计35万条。

## 数据精选方法论

为优化数据混合策略，我们遵循了[Phi-4推理技术报告](https://huggingface.co/papers/2504.21318)中提及的方法论，即可以针对每个领域独立优化数据混合比例，随后将其整合为单一数据集。在每次消融实验中，我们每轮训练都会在AIME 2024、GPQA Diamond与LiveCodeBench v4上进行评估，并选取性能最优的模型检查点（checkpoint）。

下图展示了经过后训练的[open-r1/Qwen2.5-Math-7B-RoPE-300k](https://huggingface.co/open-r1/Qwen2.5-Math-7B-RoPE-300k)在各个单独领域上的性能，与最终混合数据集的性能对比：

<img src="data_mix.png" alt="居中图片" style="display: block; margin: 0 auto;">

总体而言，我们发现同时在所有领域的数据上进行训练可获得最优效果。有关针对各领域优化数据混合策略的更多细节，请参阅下文各子章节。

> [!注意]
> 我们在消融实验中使用LiveCodeBench v4来加速评估，因为该基准集包含的问题数量约为v5的一半，但仍能代表完整基准集的性能。

### 编码领域

在开发[open-r1/OlympicCoder-7B](https://huggingface.co/open-r1/OlympicCoder-7B)的过程中，我们发现：使用C++生成R1推理轨迹在极具挑战性的[IOI 2024基准集](https://github.com/huggingface/ioi)上可获得更优性能，而Python轨迹则在LiveCodeBench（仅支持Python的基准集）上表现更佳。

因此，为优化数据混合策略，我们使用了源自[open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots)以下子集的C++与Python轨迹的混合数据：

- `solutions`：我们提示R1解决问题并生成C++代码。
- `solutions_py`：与`solutions`类似，但提示R1生成Python代码。
- `solutions_w_editorials`：我们提示R1解决问题并生成代码，同时为其提供人工撰写的题解。
- `solutions_w_editorials_py`：与`solutions_w_editorials`类似，但提示R1生成Python代码。

下图展示了我们以[Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)作为基座模型时，在这些子集上开展的消融实验的演进过程：

<img src="code_mix.png" alt="居中图片" style="display: block; margin: 0 auto;">

各次实验对应的设置如下：

* **exp1 - exp3**：将`solutions`子集的学习率从1e-5分别调整至2e-5与4e-5。
* **exp4 - exp5**：对比仅在`solutions_w_editorials`子集上训练，与同时在`solutions`和`solutions_w_editorials`子集上训练的性能差异，以评估该子集的影响。
* **exp6 - exp9**：评估引入来自`solutions_py`与`solutions_w_editorials_py`子集的Python轨迹的影响。其中exp6将`solutions_w_editorials`与`solutions_w_editorials_py`子集合并，exp7将`solutions`与`solutions_py`子集合并，exp8则合并全部四个子集。

我们发现，合并全部C++与Python轨迹子集可在LiveCodeBench上获得最优性能。同时我们发现，使用该数据混合策略对[open-r1/Qwen2.5-Coder-7B-RoPE-300k](https://huggingface.co/open-r1/Qwen2.5-Coder-7B-RoPE-300k)进行微调，可获得相当的性能提升，这证明了我们的数据精选策略的有效性。

### 数学领域

针对数学领域，我们主要对比了[open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k)的`default`与`extended`子集。其中`default`子集包含9.37万条推理轨迹，`extended`子集则额外包含13.1万条轨迹，其问题难度低于`default`子集。

下图展示了以[Qwen/Qwen2.5-Math-7B-RoPE-300k](https://huggingface.co/Qwen/Qwen2.5-Math-7B-RoPE-300k)作为基座模型时，各个子集上的性能表现：

<img src="math_mix.png" alt="居中图片" style="display: block; margin: 0 auto;">

总体而言，我们发现仅在`default`子集上训练的性能优于仅在`extended`子集上训练，而同时在两个子集上训练可获得最优性能。但考虑到同时使用两个子集将导致数据集规模大幅提升，而性能提升却较为有限，我们最终仅选择`default`子集用于最终的数据混合。

### 科学领域

针对科学领域，我们使用了[nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset/viewer/SFT/science)的`science`子集，该子集原本包含48.3万条推理轨迹。但我们发现该子集规模过大，若直接全部使用会导致数据集规模大幅攀升。

因此我们筛选出了未使用Qwen模型进行提示词预处理的轨迹子集——更多细节可参阅该[讨论帖](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset/discussions/6)。最终得到17.3万条推理轨迹，在经过学习率消融实验后，将其用于最终的数据混合。

## 引用方式

若您在研究中使用了该数据集，请按如下方式引用本数据集，以及您所使用的特定领域数据源：

bibtex
@misc{openr1,
title = {Open R1: A fully open reproduction of DeepSeek-R1},
url = {https://github.com/huggingface/open-r1},
author = {Hugging Face},
month = {January},
year = {2025}
}

**open-r1/codeforces-cots**

bibtex
@misc{penedo2025codeforces,
title={CodeForces CoTs},
author={Guilherme Penedo and Anton Lozhkov and Hynek Kydlíček and Loubna Ben Allal and Edward Beeching and Agustín Piqueres Lajarín and Quentin Gallouédec and Nathan Habib and Lewis Tunstall and Leandro von Werra},
year={2025},
publisher = {Hugging Face},
journal = {Hugging Face repository},
howpublished = {url{https://huggingface.co/datasets/open-r1/codeforces-cots}}
}

**open-r1/OpenR1-Math-220k**

bibtex
@misc{lozhkov2025openr1math220k,
title={OpenR1-Math-220k},
author={Anton Lozhkov and Hynek Kydlíček and Loubna Ben Allal and Guilherme Penedo and Edward Beeching and Quentin Gallouédec and Nathan Habib and Lewis Tunstall and Leandro von Werra},
year={2025},
publisher = {Hugging Face},
journal = {Hugging Face repository},
howpublished = {url{https://huggingface.co/datasets/open-r1/OpenR1-Math-220k}}
}

**nvidia/Llama-Nemotron-Post-Training-Dataset**

bibtex
@misc{bercovich2025llamanemotronefficientreasoningmodels,
title={Llama-Nemotron: Efficient Reasoning Models},
author={Akhiad Bercovich and Itay Levy and Izik Golan and Mohammad Dabbah and Ran El-Yaniv and Omri Puny and Ido Galil and Zach Moshe and Tomer Ronen and Najeeb Nabwani and Ido Shahaf and Oren Tropp and Ehud Karpas and Ran Zilberstein and Jiaqi Zeng and Soumye Singhal and Alexander Bukharin and Yian Zhang and Tugrul Konuk and Gerald Shen and Ameya Sunil Mahabaleshwarkar and Bilal Kartal and Yoshi Suhara and Olivier Delalleau and Zijia Chen and Zhilin Wang and David Mosallanezhad and Adi Renduchintala and Haifeng Qian and Dima Rekesh and Fei Jia and Somshubra Majumdar and Vahid Noroozi and Wasi Uddin Ahmad and Sean Narenthiran and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Siddhartha Jain and Igor Gitman and Ivan Moshkov and Wei Du and Shubham Toshniwal and George Armstrong and Branislav Kisacanin and Matvei Novikov and Daria Gitman and Evelina Bakhturina and Jane Polak Scowcroft and John Kamalu and Dan Su and Kezhi Kong and Markus Kliegl and Rabeeh Karimi and Ying Lin and Sanjeev Satheesh and Jupinder Parmar and Pritam Gundecha and Brandon Norick and Joseph Jennings and Shrimai Prabhumoye and Syeda Nahida Akter and Mostofa Patwary and Abhinav Khattar and Deepak Narayanan and Roger Waleffe and Jimmy Zhang and Bor-Yiing Su and Guyue Huang and Terry Kong and Parth Chadha and Sahil Jain and Christine Harvey and Elad Segal and Jining Huang and Sergey Kashirsky and Robert McQueen and Izzy Putterman and George Lam and Arun Venkatesan and Sherry Wu and Vinh Nguyen and Manoj Kilaru and Andrew Wang and Anna Warno and Abhilash Somasamudramath and Sandip Bhaskar and Maka Dong and Nave Assaf and Shahar Mor and Omer Ullman Argov and Scot Junkin and Oleksandr Romanenko and Pedro Larroy and Monika Katariya and Marco Rovinelli and Viji Balas and Nicholas Edelman and Anahita Bhiwandiwalla and Muthu Subramaniam and Smita Ithape and Karthik Ramamoorthy and Yuting Wu and Suguna Varshini Velury and Omri Almog and Joyjit Daw and Denys Fridman and Erick Galinkin and Michael Evans and Katherine Luna and Leon Derczynski and Nikki Pope and Eileen Long and Seth Schneider and Guillermo Siman and Tomasz Grzegorzek and Pablo Ribalta and Monika Katariya and Joey Conway and Trisha Saar and Ann Guan and Krzysztof Pawelec and Shyamala Prayaga and Oleksii Kuchaiev and Boris Ginsburg and Oluwatobi Olabiyi and Kari Briski and Jonathan Cohen and Bryan Catanzaro and Jonah Alben and Yonatan Geifman and Eric Chung and Chris Alexiuk},
year={2025},
eprint={2505.00949},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.00949},
}