synthetic-gsm8k-evolutionary-405b

# gretelai/synthetic-gsm8k-evolutionary-405b

This dataset is a synthetically generated version inspired by the GSM8K dataset, created entirely using **Gretel Navigator with meta-llama/Meta-Llama-3.1-405B** as the agent LLM. It contains Grade School-level reasoning tasks with step-by-step solutions, focusing on multi-step reasoning problems.

## Key Features:

- **Synthetically Generated**: Built using **Gretel Navigator**, leveraging evolutionary approach for diversity to create both the `question` and `answer` fields.
- **Contextual tags** ensured diversity, while **LLM-as-a-judge** was used to validate the quality of the outputs. All calculations were rigorously verified using the Python `sympy` library for accuracy.
- **Train & Test sets**: A 600-example test set is stratified by topic and difficulty.
- **Diverse Real-World Contexts**: Covers a broad range of topics, ensuring that models are trained on questions reflective of real-world scenarios.
- **Categorized by Difficulty**: Problems are organized into three difficulty levels—medium, hard, and very hard—allowing for more granular evaluation.

## Dataset Column Descriptions

* `difficulty`: The difficulty level of the problem.
* `difficulty_description`: Description of the problem's complexity and required reasoning.
* `topic`: The topic or subject of the problem.
* `context`: The context in which the problem is set.
* `age_group`: The target age or grade level for the problem.
* `culture`: The cultural background or setting reflected in the problem.
* `question`: The problem or question presented to the model.
* `answer`: The final solution to the problem.

## Dataset Statistics and Distribution


## Gretel Navigator (selected model: meta-llama/Meta-Llama-3.1-405B) Dataset - Distribution Analysis

### Topic Distribution

| topic | Train | Test |
|:-------------------------|--------:|-------:|
| algebra | 213 | 25 |
| arithmetic | 207 | 24 |
| compound interest | 167 | 20 |
| data interpretation | 224 | 27 |
| exponential growth/decay | 179 | 21 |
| fractions | 192 | 22 |
| geometry | 207 | 24 |
| optimization | 173 | 20 |
| percentages | 238 | 29 |
| polynomials | 157 | 19 |
| probability | 183 | 21 |
| proportions | 209 | 24 |
| ratios | 203 | 24 |

### Difficulty Distribution

| difficulty | Train | Test |
|:-------------|--------:|-------:|
| hard | 843 | 99 |
| medium | 969 | 113 |
| very hard | 740 | 88 |

## Citation and Usage

If you use this dataset in your research or applications, please cite it as:

```
@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-evolutionary-405b},
}
```

For questions, issues, or additional information, please visit the dataset repository on Hugging Face or contact Gretel AI.

# gretelai/synthetic-gsm8k-evolutionary-405b
本数据集为受GSM8K数据集启发生成的合成版本，完全以**Gretel Navigator（搭载meta-llama/Meta-Llama-3.1-405B）**作为智能体大语言模型（agent LLM）构建完成。其包含小学学段的推理任务与分步解答，重点聚焦多步推理类问题。

## 核心特性
- **合成生成**：依托**Gretel Navigator**构建，采用进化方法提升数据多样性，同步生成`question`（问题）与`answer`（解答）字段。
- **上下文标签**保障了数据多样性，同时采用**法官式大语言模型（LLM-as-a-judge）**验证输出质量。所有计算结果均通过Python的`sympy`库进行严格校验，以确保准确性。
- **训练集与测试集**：测试集包含600个样本，按主题与难度进行分层抽样。
- **丰富真实场景**：覆盖广泛的主题范畴，确保模型在贴合真实场景的问题上完成训练。
- **按难度分级**：问题被划分为中等（medium）、困难（hard）与极难（very hard）三个难度层级，支持更精细化的模型评估。

## 数据集字段说明
* `difficulty`：问题的难度等级。
* `difficulty_description`：对问题复杂度与所需推理能力的说明。
* `topic`：问题所属的主题或学科。
* `context`：问题的设定背景。
* `age_group`：问题对应的目标年龄段或年级水平。
* `culture`：问题所体现的文化背景或设定场景。
* `question`：向模型提出的问题或任务。
* `answer`：问题的最终解答。

## 数据集统计与分布

## 以Gretel Navigator（选用模型：meta-llama/Meta-Llama-3.1-405B）构建的数据集——分布分析

### 主题分布
| 主题 | 训练集样本数 | 测试集样本数 |
|:-------------------------|--------:|-------:|
| 代数 | 213 | 25 |
| 算术 | 207 | 24 |
| 复利 | 167 | 20 |
| 数据解读 | 224 | 27 |
| 指数增长/衰减 | 179 | 21 |
| 分数 | 192 | 22 |
| 几何 | 207 | 24 |
| 优化问题 | 173 | 20 |
| 百分比 | 238 | 29 |
| 多项式 | 157 | 19 |
| 概率 | 183 | 21 |
| 比例 | 209 | 24 |
| 比率 | 203 | 24 |

### 难度分布
| 难度等级 | 训练集样本数 | 测试集样本数 |
|:-------------|--------:|-------:|
| 困难（hard） | 843 | 99 |
| 中等（medium） | 969 | 113 |
| 极难（very hard） | 740 | 88 |

## 引用与使用说明
若您在研究或应用中使用本数据集，请按以下格式引用：

@dataset{gretelai_gsm8k_synthetic,
author = {Gretel AI},
title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B},
year = {2024},
month = {9},
publisher = {Gretel},
howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-evolutionary-405b},
}

如有疑问、问题或需获取更多信息，请访问Hugging Face上的数据集仓库或联系Gretel AI.