MetaMathQA-R1

[](https://github.com/oumi-ai/oumi)
[](https://github.com/oumi-ai/oumi)

[](https://oumi.ai/docs/en/latest/index.html)
[](https://oumi.ai/blog)
[](https://discord.gg/oumi)

# oumi-ai/MetaMathQA-R1

**MetaMathQA-R1** is a text dataset designed to train Conversational Language Models with **DeepSeek-R1 level reasoning**.
Prompts were augmented from [GSM8K](https://huggingface.co/datasets/openai/gsm8k) and [MATH](https://github.com/hendrycks/math) training sets with responses directly from **[DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)**.
**MetaMathQA-R1** was used to train **[MiniMath-R1-1.5B](https://huggingface.co/oumi-ai/MiniMath-R1-1.5B)**, which achieves **44.4% accuracy on MMLU-Pro-Math**, the **highest of any model with <=1.5B parameters**.

- **Curated by:** [Oumi AI](https://oumi.ai/) using Oumi inference on [Parasail](https://www.parasail.io/)
- **Language(s) (NLP):** English
- **License:** [Apache 2.0](https://opensource.org/license/apache-2-0)
- **Demo:** [Notebook](https://github.com/oumi-ai/oumi/blob/307436bd98706cb9ce7b0bbf31204770af2b7c8c/notebooks/Oumi%20-%20MiniMath-R1-1.5B.ipynb)

## Uses

<!-- This section describes suitable use cases for the dataset. -->
Use this dataset for supervised fine-tuning of LLMs for improved performance in math by enabling inference-time reasoning. Can also be incorporated into a training mixture for creating an R1-like model.

## Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->
This dataset is not well suited for producing generalized chat models on its own due to being entirely math queries.

## Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->
```
{
# Unique conversation identifier, tied back to MetaMathQA samples
"conversation_id": str,

# The user turn/prompt
"prompt": str,

# The assistant (DeepSeek R1) response
# Includes the thought trace which is wrapped in <think> and </think> tags
"response": str,

# Data formatted to user + assistant turns in chat format
# Example: [{'role': 'user', 'content': ...}, {'role': 'assistant', 'content': ...}]
"messages": list[dict[str, str]],

# Metadata for sample
"metadata": dict[str, ...],
}
```

## Dataset Creation

### Curation Rationale

<!-- Motivation for the creation of this dataset. -->
To enable the community to develop a fully-open state-of-the-art Foundational Language Model, we've produced and released this dataset to serve as part of the foundation of reasoning data for the model. It was produced using the Oumi’s inference capabilities on Parasail.

### Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->
Queries were sourced from [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA) which is data augmented from the training sets of GSM8K and MATH.

#### Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->
* Responses were collected via Oumi's batch inference support for [Parasail](https://parasail.io/).
* Samples which could not be parsed were discarded (<100).
* All other samples include metadata indicating if they are complete or not (which was determined by whether or not a `</think>` token is present)

#### Personal and Sensitive Information

<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. -->
Data is not known or likely to contain any personal, sensitive, or private information.

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

1. The source prompts are from [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA) `query` column and may reflect any biases in their data augmentation process.
2. The responses produced will likely be reflective of any biases or limitations produced by DeepSeek-R1.

## Citation

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```
@misc{metaMathQaR12025,
author = {Jeremiah Greer},
title = {MetaMathQA-R1 Dataset},
month = {February},
year = {2025},
url = {https://huggingface.co/datasets/oumi-ai/MetaMathQA-R1}
}

@software{oumi2025,
author = {Oumi Community},
title = {Oumi: an Open, End-to-end Platform for Building Large Foundation Models},
month = {January},
year = {2025},
url = {https://github.com/oumi-ai/oumi}
}
```

[](https://github.com/oumi-ai/oumi)
[](https://github.com/oumi-ai/oumi)

[](https://oumi.ai/docs/en/latest/index.html)
[](https://oumi.ai/blog)
[](https://discord.gg/oumi)

# oumi-ai/MetaMathQA-R1

**MetaMathQA-R1** 是一款专为训练具备**DeepSeek-R1 级推理能力**的对话语言模型设计的文本数据集。其提示词源自[GSM8K](https://huggingface.co/datasets/openai/gsm8k)与[MATH](https://github.com/hendrycks/math)训练集的增强版数据，响应内容则直接来自**[DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)**。**MetaMathQA-R1** 曾用于训练**[MiniMath-R1-1.5B](https://huggingface.co/oumi-ai/MiniMath-R1-1.5B)**，该模型在**MMLU-Pro-Math**上实现了44.4%的准确率，为参数量≤15亿的模型中的最高水平。

- **整理方：** [Oumi AI](https://oumi.ai/)，依托Parasail平台的Oumi推理服务完成制作
- **NLP所用语言：** 英语
- **许可协议：** [Apache 2.0](https://opensource.org/license/apache-2-0)
- **演示示例：** [Notebook](https://github.com/oumi-ai/oumi/blob/307436bd98706cb9ce7b0bbf31204770af2b7c8c/notebooks/Oumi%20-%20MiniMath-R1-1.5B.ipynb)

## 使用场景
<!-- 本部分描述了该数据集的适用场景。 -->
本数据集适用于对大语言模型进行监督微调，以提升其数学推理性能，支持推理阶段的思维链生成。也可被纳入训练混合数据集，用于构建类似R1的推理模型。

## 不适用场景
<!-- 本部分阐述了误用、恶意使用以及本数据集无法良好适配的使用场景。 -->
由于本数据集仅包含数学查询类内容，单独使用本数据集无法训练得到通用对话模型。

## 数据集结构
<!-- 本部分描述了数据集的字段信息，以及数据集划分标准、样本间关系等额外结构信息。 -->

{
# 唯一对话标识符，关联至MetaMathQA样本
"conversation_id": str,

# 用户轮次的提示词
"prompt": str,

# 助手（DeepSeek R1）的响应内容
# 包含包裹在<think>与</think>标签内的思维链痕迹
"response": str,

# 按照用户+助手轮次格式化的对话数据
# 示例：[{'role': 'user', 'content': ...}, {'role': 'assistant', 'content': ...}]
"messages": list[dict[str, str]],

# 样本元数据
"metadata": dict[str, ...],
}

## 数据集构建
### 整理初衷
<!-- 本数据集的创建动机。 -->
为助力社区开发完全开源的顶尖基础大语言模型，我们制作并发布本数据集，作为模型推理训练数据的核心组成部分。本数据集依托Parasail平台的Oumi推理能力完成制作。

### 源数据
<!-- 本部分描述了源数据的来源，例如新闻文本与标题、社交媒体帖文、翻译句对等。 -->
查询问题源自[MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA)，而该数据集本身是基于GSM8K与MATH训练集增强得到的。

#### 数据收集与处理流程
<!-- 本部分描述了数据收集与处理的具体流程，包括数据筛选标准、过滤与归一化方法、所用工具与库等。 -->
* 响应内容通过Oumi在Parasail平台的批量推理服务获取。
* 无法正常解析的样本已被剔除（数量少于100）。
* 其余所有样本均包含元数据，用于标记样本是否完整（判断依据为是否存在`</think>`标记）。

#### 个人与敏感信息
<!-- 说明数据集是否包含可能被视为个人、敏感或隐私的数据，例如涉及地址、唯一可识别的姓名或别名、种族或族裔起源、性取向、宗教信仰、政治观点、财务或健康数据等。若已采取匿名化措施，请描述该流程。 -->
本数据集未包含任何已知或疑似的个人、敏感或私有信息。

## 偏差、风险与局限性
<!-- 本部分旨在阐述技术与社会技术层面的局限性。 -->
1. 源提示词取自[MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA)的`query`字段，可能继承其数据增强过程中引入的各类偏差。
2. 生成的响应大概率会反映DeepSeek-R1本身存在的偏差与局限性。

## 引用信息
<!-- 若有介绍该数据集的论文或博客文章，需在此处附上APA与Bibtex格式的引用信息。 -->

**BibTeX格式：**

@misc{metaMathQaR12025,
author = {Jeremiah Greer},
title = {MetaMathQA-R1 数据集},
month = {February},
year = {2025},
url = {https://huggingface.co/datasets/oumi-ai/MetaMathQA-R1}
}

@software{oumi2025,
author = {Oumi Community},
title = {Oumi：开源端到端大基础模型构建平台},
month = {January},
year = {2025},
url = {https://github.com/oumi-ai/oumi}
}