gretel-math-gsm8k-v1

<center>
<img src="https://cdn-uploads.huggingface.co/production/uploads/632ca8dcdbea00ca213d101a/Dxczp-90GBrGgDMDExpdv.png" alt="gretelai/synthetic-gsm8k-reflection-405b" width="600px">
<p><em>Image generated by DALL-E. See <a href="https://huggingface.co/datasets/gretelai/synthetic_text_to_sql/blob/main/dalle_prompt.txt">prompt</a> for more details</em></p>
</center>

# gretelai/synthetic-gsm8k-reflection-405b

This dataset is a synthetically generated version inspired by the GSM8K dataset, created entirely using **Gretel Navigator with meta-llama/Meta-Llama-3.1-405B** as the agent LLM. It contains Grade School-level reasoning tasks with step-by-step reflections and solutions, focusing on multi-step reasoning problems.

## Key Features for AI Developers:

- **Synthetic Data Generation**: Data created using **Gretel Navigator**, including automated output validation and quality evaluations.
- **Reflection Prompts**: Structured reasoning captured using `<thinking>`, `<reflection>`, and `<output>` tags, detailing the AI's decision-making process.
- **Evaluation**: Outputs validated by **LLM-as-a-judge** to ensure quality and consistency.
- **Validation**: Calculation annotations verified for accuracy using the Python `sympy` library.
- **Diverse Real-World Contexts**: Dataset covers a broad range of topics, providing realistic scenarios for natural language reasoning.
- **Contextual Tags**: Applied to ensure diversity in data, helping the model generalize across different question types.
- **Difficulty Levels**: Problems organized into four levels—easy, medium, hard, and very hard—offering increasing complexity beyond the original `gsm8k` dataset.
- **Train & Test Sets**: Includes a 1300-example test set, stratified by topic and difficulty for evaluation.

## Dataset Column Descriptions

* `difficulty`: The difficulty level of the problem.
* `difficulty_description`: Description of the problem's complexity and required reasoning.
* `topic`: The topic or subject of the problem.
* `context`: The context in which the problem is set.
* `age_group`: The target age or grade level for the problem.
* `question`: The problem or question presented to the model.
* `answer`: The final solution to the problem.
* `answer_with_tags`: The step-by-step thought process, including thinking and reflection.

## Dataset Statistics and Distribution


## Gretel Navigator (selected model: meta-llama/Meta-Llama-3.1-405B) Dataset - Distribution Analysis

### Topic Distribution

| topic | Train | Test |
|:-------------------------|--------:|-------:|
| algebra | 1871 | 104 |
| arithmetic | 2319 | 128 |
| compound interest | 1544 | 86 |
| data interpretation | 1822 | 100 |
| exponential growth/decay | 1702 | 93 |
| fractions | 1739 | 96 |
| geometry | 1897 | 105 |
| optimization | 1463 | 80 |
| percentages | 2587 | 143 |
| polynomials | 980 | 54 |
| probability | 1809 | 100 |
| proportions | 1978 | 108 |
| ratios | 1867 | 103 |

### Difficulty Distribution

| difficulty | Train | Test |
|:-------------|--------:|-------:|
| easy | 6608 | 365 |
| hard | 5054 | 280 |
| medium | 6765 | 373 |
| very hard | 5151 | 282 |

## Citation and Usage

If you use this dataset in your research or applications, please cite it as:

```
@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-reflection-405b},
}
```

For questions, issues, or additional information, please visit the dataset repository on Hugging Face or contact Gretel AI.

<center>
<img src="https://cdn-uploads.huggingface.co/production/uploads/632ca8dcdbea00ca213d101a/Dxczp-90GBrGgDMDExpdv.png" alt="gretelai/synthetic-gsm8k-reflection-405b" width="600px">
<p><em>图片由DALL-E生成。详细提示词请参见<a href="https://huggingface.co/datasets/gretelai/synthetic_text_to_sql/blob/main/dalle_prompt.txt">此处</a></em></p>
</center>

# gretelai/synthetic-gsm8k-reflection-405b

本数据集是受GSM8K（GSM8K）数据集启发而合成生成的版本，完全通过以**Gretel Navigator搭配meta-llama/Meta-Llama-3.1-405B**作为代理大语言模型（LLM）构建而成。数据集包含中小学水平的推理任务，附带逐步骤的反思与解答，聚焦于多步推理类问题。

## 面向AI开发者的核心特性：

- **合成数据生成**：基于**Gretel Navigator**构建数据，包含自动化输出验证与质量评估环节。
- **反思提示设计**：通过`<thinking>`、`<reflection>`与`<output>`标签捕获结构化推理过程，详细呈现AI的决策逻辑。
- **评估机制**：采用**LLM作为评判者（LLM-as-a-judge）**对输出结果进行验证，确保内容质量与一致性。
- **验证环节**：通过Python的`sympy`库对计算类标注进行准确性校验。
- **多样化真实场景**：数据集涵盖广泛的主题范畴，为自然语言推理任务提供贴合现实的场景。
- **上下文标签**：通过添加上下文标签保障数据多样性，助力模型在各类题型中实现泛化能力。
- **难度分级体系**：将题目划分为四个难度等级——简单、中等、困难、极困难，相比原始`gsm8k`数据集，提供了复杂度逐步提升的推理任务。
- **训练集与测试集**：包含1300条样本的测试集，按主题与难度分层构建，适配模型评估需求。

## 数据集字段说明：

* `difficulty`：题目难度等级。
* `difficulty_description`：题目复杂度与所需推理能力的说明。
* `topic`：题目所属主题或学科。
* `context`：题目设定的场景背景。
* `age_group`：题目适配的目标年龄或年级水平。
* `question`：向模型提出的问题或任务。
* `answer`：题目的最终解答。
* `answer_with_tags`：包含思考与反思环节的逐步骤推理过程。

## 数据集统计与分布


## 基于Gretel Navigator（选用模型：meta-llama/Meta-Llama-3.1-405B）的数据集分布分析

### 主题分布

| 主题 | 训练集样本数 | 测试集样本数 |
|:-------------------------|--------:|-------:|
| 代数 | 1871 | 104 |
| 算术 | 2319 | 128 |
| 复利 | 1544 | 86 |
| 数据解读 | 1822 | 100 |
| 指数增长/衰减 | 1702 | 93 |
| 分数 | 1739 | 96 |
| 几何 | 1897 | 105 |
| 优化 | 1463 | 80 |
| 百分比 | 2587 | 143 |
| 多项式 | 980 | 54 |
| 概率 | 1809 | 100 |
| 比例 | 1978 | 108 |
| 比率 | 1867 | 103 |

### 难度分布

| 难度等级 | 训练集样本数 | 测试集样本数 |
|:-------------|--------:|-------:|
| 简单 | 6608 | 365 |
| 困难 | 5054 | 280 |
| 中等 | 6765 | 373 |
| 极困难 | 5151 | 282 |

## 引用与使用说明

若您在研究或应用中使用本数据集，请按以下格式引用：

@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-reflection-405b},
}

如有疑问、问题或需要更多信息，请访问Hugging Face上的数据集仓库或联系Gretel AI。