sapienzanlp/gsm8k_italian

--- license: mit task_categories: - text-generation language: - it - en size_categories: - 1K<n<10K configs: - config_name: default data_files: - split: train path: gsm8k.train.json - split: test path: gsm8k.test.json --- # GSM8K - Italian (IT) This dataset is an Italian translation of [GSM8K](https://arxiv.org/abs/2110.14168). GSM8K stands for Grade School Math 8K, a dataset for math word problems, which should be easy to solve for people with an elementary school education. ## Dataset Details The dataset consists of math word problems, where each problem is associated with a possible explanation of how to solve it. The task is to generate the answer to the math problem. The dataset is split into a training set and a test set. The training set contains 7,000 examples, while the test set contains 1,000 examples. The dataset includes the following splits from the `main` configuration of the original dataset: * Train: 7,473 rows * Test: 1,319 rows ### Differences with the original dataset * The answers in the original dataset include the full explanation of how to solve the problem, including a set of "annotations" between angular brackets. In this version, we have separated the answer from the annotations (see provided examples for more details). * We also provide 4 distractors for each question to make the dataset a multiple-choice question answering dataset too. * The number of instances in this dataset is smaller than the original dataset due to the translation process, during which some instances were filtered out. ### Languages This dataset is **fully parallel** between English and Italian. This allows us to have comparable evaluation setups and results across the two languages. ### Translation Process The translation has been carried out using [🍱 OBenTO-LLM](https://github.com/c-simone/llm-data-translation), an open-source tool for LLM-based translation. The main motivation for using an open-source LLM is to encourage free, open, reproducible, and transparent research in LLM evaluation. See [🍱 OBenTO-LLM](https://github.com/c-simone/llm-data-translation) for more details on the translation process. **Model used to translate:** [Unbabel/TowerInstruct-7B-v0.2] (https://huggingface.co/Unbabel/TowerInstruct-7B-v0.2) ### Other Information - **Original dataset by:** [Cobbe et al.](https://arxiv.org/abs/2110.14168) - **Translation by:** [Simone Conia](https://scholar.google.com/citations?user=S1tqbTcAAAAJ) - **Languages:** Italian, English - **License:** MIT ## Dataset Format This is an example that shows the format of the dataset, where: * `id`: a unique ID for each sample; * `category`: type of task; * `input_text`: the original English sentence in the dataset; * `input_text_translation`: the translation of the sentence in Italian; * `choices`: the original English choices; * `choice_translations`: the translation of the choices in Italian; * `label`: the index of the correct answer. * `metadata`: additional information about the question, including the correct answer, the explanation of how to solve the problem, and the annotated explanation. #### Example of a question in GSM8K ```json { "id": "gsm8k_1", "category": "question", "input_text": "Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How much did she earn?", "input_text_translation": "Weng guadagna 12 dollari all'ora per fare la babysitter. Ieri ha fatto 50 minuti di babysitter. Quanto ha guadagnato?", "choices": [ 10, 8, 9, 11, 12 ], "choice_translations": [ 10, 8, 9, 11, 12 ], "label": 0, "metadata": { "result": 10, "explanation": "Weng earns 12/60 = $0.2 per minute. Working 50 minutes, she earned 0.2 x 50 = $10.", "explanation_translation": "Weng guadagna 12/60 = 0,2 $ al minuto. Lavorando 50 minuti, ha guadagnato 0,2 x 50 = 10 $.", "annotated_explanation": "Weng earns 12/60 = $<<12/60=0.2>>0.2 per minute. Working 50 minutes, she earned 0.2 x 50 = $<<0.2*50=10>>10.", "annotated_explanation_translation": "Weng guadagna 12/60 = $<<12/60=0.2>>0.2 al minuto. Lavorando 50 minuti, ha guadagnato 0.2 x 50 = $<<0.2*50=10>>10." } } ``` ## License The dataset is distributed under the MIT license. ## Acknowledgements I would like to thank the authors of the original dataset for making it available to the research community. I would also like to thank [Future AI Research](https://future-ai-research.it/) for supporting this work and funding my research. ### Special Thanks My special thanks go to: * Pere-Lluís Huguet Cabot and Riccardo Orlando for their help with [🍱 OBenTO-LLM](https://github.com/c-simone/llm-data-translation). ## Dataset Card Authors * [Simone Conia](https://scholar.google.com/citations?user=S1tqbTcAAAAJ): [e-mail] dataset is an Italian translation of GSM8K, which stands for Grade School Math 8K. It consists of math word problems designed for elementary school education level. The dataset includes both English and Italian versions, making it fully parallel. It is split into training and test sets with 7,473 and 1,319 rows respectively. The translation process used an open-source tool called OBenTO-LLM to ensure transparency and reproducibility. The dataset format includes fields such as id, category, input_text, input_text_translation, choices, choice_translations, label, and metadata.

原始信息汇总

GSM8K - Italian (IT)

数据集概述

• 任务类别: 文本生成
• 语言: 意大利语, 英语
• 数据集大小: 1K<n<10K
• 配置:
  • 默认配置:
    • 训练集: gsm8k.train.json
    • 测试集: gsm8k.test.json

数据集详情

• 数据类型: 数学应用题
• 任务: 生成数学问题的答案
• 数据集分割:
  • 训练集: 7,000 条数据
  • 测试集: 1,000 条数据
• 数据集版本:
  • 训练集: 7,473 行
  • 测试集: 1,319 行

与原始数据集的差异

• 答案分离: 将答案与解释分离
• 多选题: 提供4个干扰选项
• 数据量减少: 由于翻译过程，数据量少于原始数据集

语言

• 完全平行: 英语和意大利语

翻译过程

• 工具: 🍱 OBenTO-LLM

数据集格式

• 字段:
  • id: 唯一ID
  • category: 任务类型
  • input_text: 原始英语句子
  • input_text_translation: 意大利语翻译
  • choices: 原始英语选项
  • choice_translations: 意大利语选项
  • label: 正确答案的索引
  • metadata: 附加信息，包括正确答案、解释和注释解释

许可证

• MIT许可证