synthetic-gsm8k-evolutionary-405b
收藏资源简介：
该数据集是一个合成生成的版本，灵感来源于GSM8K数据集，完全使用Gretel Navigator和meta-llama/Meta-Llama-3.1-405B作为代理LLM创建。它包含小学级别的推理任务，具有逐步解决方案，专注于多步推理问题。数据集通过Gretel Navigator利用进化方法生成多样性，确保了问题和答案字段的多样性。所有计算都使用Python的sympy库进行了严格验证。数据集包含600个示例的测试集，按主题和难度分层。问题涵盖了广泛的主题，确保模型在反映现实世界场景的问题上进行训练。问题按难度分为三个级别：中等、困难和非常困难，允许进行更细粒度的评估。数据集的列包括难度、难度描述、主题、上下文、年龄组、文化、问题和答案。
This is a synthetic dataset inspired by the GSM8K dataset, fully created using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B as the proxy LLMs. It contains primary-school-level reasoning tasks with step-by-step solutions, focusing on multi-step reasoning problems. The dataset leverages evolutionary methods via Gretel Navigator to generate diversity, ensuring variety in both the question and answer fields. All calculations are rigorously verified using Python's sympy library. The dataset includes a test set of 600 examples, stratified by topic and difficulty. The questions cover a wide range of topics, ensuring that models are trained on problems that reflect real-world scenarios. The problems are categorized into three difficulty levels: medium, hard, and very hard, enabling more fine-grained evaluation. The dataset's columns include difficulty, difficulty description, topic, context, age group, culture, question, and answer.
gretelai/synthetic-gsm8k-evolutionary-405b
概述
- 语言: 英语
- 许可: llama3.1
- 多语言性: 单语种
- 数据集大小: 1K<n<10K
- 源数据集: 原始数据集
- 任务类别: 问答
- 任务ID: 封闭领域问答
- PapersWithCode ID: gsm8k
关键特性
- 合成生成: 使用 Gretel Navigator 生成，采用进化方法确保多样性，生成
question和answer字段。 - 上下文标签: 确保多样性，使用 LLM-as-a-judge 验证输出质量，所有计算通过 Python
sympy库严格验证。 - 训练与测试集: 包含600个示例的测试集，按主题和难度分层。
- 多样化的现实世界情境: 涵盖广泛的主题，确保模型训练的问题反映现实世界场景。
- 按难度分类: 问题分为三个难度级别——中等、困难和非常困难，允许更细粒度的评估。
数据集列描述
difficulty: 问题的难度级别。difficulty_description: 问题的复杂性和所需推理的描述。topic: 问题的主题或学科。context: 问题设置的上下文。age_group: 问题的目标年龄或年级。culture: 问题中反映的文化背景或环境。question: 提供给模型的问题或问题。answer: 问题的最终解决方案。
数据集统计和分布
-
主题分布:
topic Train Test algebra 213 25 arithmetic 207 24 compound interest 167 20 data interpretation 224 27 exponential growth/decay 179 21 fractions 192 22 geometry 207 24 optimization 173 20 percentages 238 29 polynomials 157 19 probability 183 21 proportions 209 24 ratios 203 24 -
难度分布:
difficulty Train Test hard 843 99 medium 969 113 very hard 740 88
引用和使用
-
引用:
@dataset{gretelai_gsm8k_synthetic, author = {Gretel AI}, title = {Synthetically Generated Reasoning Dataset (GSM8k-inspired) with enhanced diversity using Gretel Navigator and meta-llama/Meta-Llama-3.1-405B}, year = {2024}, month = {9}, publisher = {Gretel}, howpublished = {https://huggingface.co/gretelai/synthetic-gsm8k-evolutionary-405b}, }
