synthetic-gsm8k-reflection-405b

<center>
<img src="https://cdn-uploads.huggingface.co/production/uploads/632ca8dcdbea00ca213d101a/Dxczp-90GBrGgDMDExpdv.png" alt="gretelai/synthetic-gsm8k-reflection-405b" width="600px">
<p><em>Image generated by DALL-E. See <a href="https://huggingface.co/datasets/gretelai/synthetic_text_to_sql/blob/main/dalle_prompt.txt">prompt</a> for more details</em></p>
</center>

# gretelai/synthetic-gsm8k-reflection-405b

This dataset is a synthetically generated version inspired by the GSM8K dataset, created entirely using **Gretel Navigator with meta-llama/Meta-Llama-3.1-405B** as the agent LLM. It contains Grade School-level reasoning tasks with step-by-step reflections and solutions, focusing on multi-step reasoning problems.

## Key Features for AI Developers:

- **Synthetic Data Generation**: Data created using **Gretel Navigator**, including automated output validation and quality evaluations.
- **Reflection Prompts**: Structured reasoning captured using `<thinking>`, `<reflection>`, and `<output>` tags, detailing the AI's decision-making process.
- **Evaluation**: Outputs validated by **LLM-as-a-judge** to ensure quality and consistency.
- **Validation**: Calculation annotations verified for accuracy using the Python `sympy` library.
- **Diverse Real-World Contexts**: Dataset covers a broad range of topics, providing realistic scenarios for natural language reasoning.
- **Contextual Tags**: Applied to ensure diversity in data, helping the model generalize across different question types.
- **Difficulty Levels**: Problems organized into four levels—easy, medium, hard, and very hard—offering increasing complexity beyond the original `gsm8k` dataset.
- **Train & Test Sets**: Includes a 1300-example test set, stratified by topic and difficulty for evaluation.

## Dataset Column Descriptions

* `difficulty`: The difficulty level of the problem.
* `difficulty_description`: Description of the problem's complexity and required reasoning.
* `topic`: The topic or subject of the problem.
* `context`: The context in which the problem is set.
* `age_group`: The target age or grade level for the problem.
* `question`: The problem or question presented to the model.
* `answer`: The final solution to the problem.
* `answer_with_tags`: The step-by-step thought process, including thinking and reflection.

## Dataset Statistics and Distribution


## Gretel Navigator (selected model: meta-llama/Meta-Llama-3.1-405B) Dataset - Distribution Analysis

### Topic Distribution

| topic | Train | Test |
|:-------------------------|--------:|-------:|
| algebra | 1871 | 104 |
| arithmetic | 2319 | 128 |
| compound interest | 1544 | 86 |
| data interpretation | 1822 | 100 |
| exponential growth/decay | 1702 | 93 |
| fractions | 1739 | 96 |
| geometry | 1897 | 105 |
| optimization | 1463 | 80 |
| percentages | 2587 | 143 |
| polynomials | 980 | 54 |
| probability | 1809 | 100 |
| proportions | 1978 | 108 |
| ratios | 1867 | 103 |

### Difficulty Distribution

| difficulty | Train | Test |
|:-------------|--------:|-------:|
| easy | 6608 | 365 |
| hard | 5054 | 280 |
| medium | 6765 | 373 |
| very hard | 5151 | 282 |

## Citation and Usage

If you use this dataset in your research or applications, please cite it as:

```
@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-reflection-405b},
}
```

For questions, issues, or additional information, please visit the dataset repository on Hugging Face or contact Gretel AI.

<center>
<img src="https://cdn-uploads.huggingface.co/production/uploads/632ca8dcdbea00ca213d101a/Dxczp-90GBrGgDMDExpdv.png" alt="gretelai/synthetic-gsm8k-reflection-405b" width="600px">
<p><em>该图片由DALL-E生成，如需了解更多细节，请查看<a href="https://huggingface.co/datasets/gretelai/synthetic_text_to_sql/blob/main/dalle_prompt.txt">提示词</a></em></p>
</center>

# gretelai/synthetic-gsm8k-reflection-405b

本数据集为受GSM8K数据集启发而合成生成的版本，完全以**Gretel Navigator**搭配meta-llama/Meta-Llama-3.1-405B作为智能体大语言模型构建而成。数据集包含小学阶段的推理任务，附带逐步骤的反思与解答，重点聚焦于多步推理问题。

## 面向AI开发者的核心特性:

- **合成数据生成**：基于**Gretel Navigator**生成数据，包含自动化输出验证与质量评估流程。
- **反思提示词**：通过`<thinking>`, `<reflection>`和`<output>`标签捕获结构化推理过程，详细阐述AI的决策逻辑。
- **评估环节**：采用**大语言模型作为评判者（LLM-as-a-judge）**对输出结果进行验证，确保质量与一致性。
- **验证环节**：通过Python的`sympy`库对计算注解的准确性进行校验。
- **多样化真实场景**：数据集涵盖广泛的主题领域，为自然语言推理任务提供贴合现实的场景。
- **上下文标签**：通过添加标签保障数据多样性，助力模型在各类题型上实现泛化能力。
- **难度分级**：将问题划分为简单、中等、困难、极难四个等级，相比原始`gsm8k`数据集，提供了复杂度逐步提升的任务。
- **训练集与测试集**：包含1300个样本的测试集，且按照主题与难度进行分层，便于开展模型评估。

## 数据集字段说明

* `difficulty`：问题的难度等级。
* `difficulty_description`：对问题复杂度与所需推理能力的描述。
* `topic`：问题所属的主题或学科。
* `context`：问题设定的上下文场景。
* `age_group`：问题适配的目标年龄或年级水平。
* `question`：向模型提出的问题或任务。
* `answer`：问题的最终解答方案。
* `answer_with_tags`：包含思考与反思环节的分步推理过程。

## 数据集统计与分布


## 基于Gretel Navigator（选定模型：meta-llama/Meta-Llama-3.1-405B）的数据集——分布分析

### 主题分布

| 主题分类 | 训练集样本数 | 测试集样本数 |
|:------------------------|------------:|------------:|
| 代数 | 1871 | 104 |
| 算术 | 2319 | 128 |
| 复利计算 | 1544 | 86 |
| 数据解读 | 1822 | 100 |
| 指数增长/衰减 | 1702 | 93 |
| 分数运算 | 1739 | 96 |
| 几何 | 1897 | 105 |
| 优化问题 | 1463 | 80 |
| 百分比计算 | 2587 | 143 |
| 多项式 | 980 | 54 |
| 概率 | 1809 | 100 |
| 比例计算 | 1978 | 108 |
| 比率计算 | 1867 | 103 |

### 难度分布

| 难度等级 | 训练集样本数 | 测试集样本数 |
|:-----------|------------:|------------:|
| 简单 | 6608 | 365 |
| 困难 | 5054 | 280 |
| 中等 | 6765 | 373 |
| 极难 | 5151 | 282 |

## 引用与使用说明

若在研究或应用中使用本数据集，请按以下格式引用：

@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-reflection-405b},
}

如需咨询问题、反馈议题或获取更多信息，请访问Hugging Face上的数据集仓库或联系Gretel AI。