ljvmiranda921/PolyglotTeachers-SFT-Synth

--- dataset_info: features: - name: id dtype: string - name: source dtype: string - name: language dtype: string - name: strategy dtype: string - name: source_id dtype: string - name: synth_prompt dtype: string - name: model dtype: string - name: prompt dtype: string - name: response dtype: string - name: messages list: - name: content dtype: string - name: role dtype: string splits: - name: train num_bytes: 2326387825 num_examples: 356471 download_size: 1083096690 dataset_size: 2326387825 configs: - config_name: default data_files: - split: train path: data/train-* language: - ar - de - id - ja - es - cs - tl license: apache-2.0 task_categories: - text-generation tags: - multilingual - synthetic - sft pretty_name: PolyglotTeachers-SFT (Synthetic) --- <img alt="Logo for LTL" src="ltl_logo2.svg" width="240px" style="margin-left:'auto' margin-right:'auto' display:'block'"> # PolyglotTeachers-SFT-Synth This dataset contains synthetic supervised fine-tuning examples generated by the best teacher we found in the paper [Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation](), where we systematically characterize what makes a good teacher model. It contains examples across six languages: Arabic, Czech, German, Indonesian, Japanese, Spanish, and Tagalog. **Note:** In our experiments, we subsampled 10k examples per language for training. Here we release the full unfiltered set to enable reproducibility and give researchers the flexibility to construct their own subsamples or training mixtures. ## Dataset Summary - **Languages:** Arabic (ar), Czech (cs), German (de), Indonesian (id), Japanese (ja), Spanish (es), Tagalog (tl) - **Total examples:** 315,596 - **Teacher model:** [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it) - **Generation strategies:** generate, respond, translate ### Language Distribution | Language | Examples | |----------|----------| | Indonesian (id) | 85,952 | | German (de) | 83,878 | | Arabic (ar) | 77,770 | | Japanese (ja) | 27,198 | | Tagalog (tl) | 40,875 | | Spanish (es) | 25,609 | | Czech (cs) | 15,189 | ## Data Sources The seed data comes from several multilingual datasets, which were then used to synthesize new examples via Gemma-3-27B-IT. Each source dataset was processed using one of three strategies: **generate** (create new prompt-response pairs from a seed), **respond** (generate a response given a prompt), or **translate** (translate an English example into a target language). * [allenai/WildChat-4.8M](https://huggingface.co/datasets/allenai/WildChat-4.8M): multilingual prompt-response pairs from real user interactions. * [openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k): math word problems (English, translated into target languages). * [Magpie-Align/Magpie-Pro-300K-Filtered](https://huggingface.co/datasets/Magpie-Align/Magpie-Pro-300K-Filtered): general chat data (English, translated into target languages). * [nvidia/Helpsteer3](https://huggingface.co/datasets/nvidia/Helpsteer3): multilingual preference data. * [OpenAssistant/oasst2](https://huggingface.co/datasets/OpenAssistant/oasst2): multilingual assistant conversations. * [utter-project/EuroBlocks-SFT-Synthetic-1124](https://huggingface.co/datasets/utter-project/EuroBlocks-SFT-Synthetic-1124): European multilingual synthetic data. * [CohereLabs/aya_collection](https://huggingface.co/datasets/CohereLabs/aya_collection): multilingual instruction data. * [arbml/CIDAR](https://huggingface.co/datasets/arbml/CIDAR): Arabic instruction data. * [indonlp/cendol_collection_v2](https://huggingface.co/datasets/indonlp/cendol_collection_v2): Indonesian instruction data. ## Dataset Structure Each example contains the following fields: | Field | Type | Description | |-------|------|-------------| | `id` | str | Unique identifier | | `source` | str | Source dataset name | | `language` | str | ISO 639-1 language code | | `strategy` | str | Synthesis strategy used (`generate`, `respond`, or `translate`) | | `source_id` | str | Identifier from the source dataset | | `synth_prompt` | str | The prompt used to instruct the teacher model during synthesis | | `model` | str | Teacher model used for generation | | `prompt` | str | The user prompt | | `response` | str | The model response | | `messages` | list | Chat-formatted messages (`role` and `content`) for SFT | ## Usage ```python from datasets import load_dataset ds = load_dataset("ljvmiranda921/PolyglotTeachers-SFT-Synth", split="train") # Filter by language arabic_ds = ds.filter(lambda x: x["language"] == "ar") # Use the messages field directly for SFT print(arabic_ds[0]["messages"]) ``` ## Acknowledgements LJVM and AK acknowledge the support of the UKRI Frontier Grant EP/Y031350/1 ([EQUATE](https://gtr.ukri.org/projects?ref=EP%2FY031350%2F1)). This work was performed using joint resources provided by the [Cambridge Service for Data Driven Discovery (CSD3)](https://hpc.cam.ac.uk/high-performance-computing) EP/T022159/1 and the [Isambard AI National AI Research Resource (AIRR)](https://www.bristol.ac.uk/research/centres/bristol-supercomputing/#isambard-ai) ST/AIRR/I-A-I/1023, and the Microsoft Research Grant. LJVM would also like to thank Songbo Hu, Chen Cecilia Liu, Millicent Ochieng, and Felermino Ali for helpful and productive discussions on the project. ## Citation ```bibtex @misc{miranda2026polyglotteachersevaluatinglanguage, title={Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation}, author={Lester James V. Miranda and Ivan Vulić and Anna Korhonen}, year={2026}, eprint={2604.11290}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2604.11290}, } ```数据集信息： 特征字段： - 名称：id，数据类型：字符串 - 名称：source，数据类型：字符串 - 名称：language，数据类型：字符串 - 名称：strategy，数据类型：字符串 - 名称：source_id，数据类型：字符串 - 名称：synth_prompt，数据类型：字符串 - 名称：model，数据类型：字符串 - 名称：prompt，数据类型：字符串 - 名称：response，数据类型：字符串 - 名称：messages，列表类型： - 名称：content，数据类型：字符串 - 名称：role，数据类型：字符串 划分集： - 名称：train，字节数：2326387825，样本数：356471 下载大小：1083096690，数据集总大小：2326387825 配置项： - 配置名称：default，数据文件： - 划分集：train，路径：data/train-* 支持语言： - ar - de - id - ja - es - cs - tl 授权协议：apache-2.0 任务类别： - 文本生成 标签： - 多语言 - 合成式数据 - 监督微调（SFT） 展示名称：PolyglotTeachers-SFT（合成式） --- <img alt="LTL 标志" src="ltl_logo2.svg" width="240px" style="margin-left:auto; margin-right:auto; display:block;"> # PolyglotTeachers-SFT-Synth 本数据集包含由论文《Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation》中验证的最优教师模型生成的合成式监督微调（supervised fine-tuning, SFT）样本，该论文系统阐释了优质教师模型的核心特征。 本数据集涵盖阿拉伯语、捷克语、德语、印尼语、日语、西班牙语及他加禄语共7种语言的样本。**注意：** 我们在实验中为每种语言采样10k条样本用于训练。本次我们发布全部未过滤的完整数据集，以保障研究可复现性，并为研究者提供灵活构建自定义采样子集或训练混合集的空间。 ## 数据集概览 - **覆盖语言**：阿拉伯语（ar）、捷克语（cs）、德语（de）、印尼语（id）、日语（ja）、西班牙语（es）、他加禄语（tl） - **总样本数**：315,596 - **教师模型**：[google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it) - **生成策略**：生成（generate）、响应（respond）、翻译（translate） ### 语言分布 | 语言 | 样本数量 | |------------|----------| | 印尼语（id） | 85,952 | | 德语（de） | 83,878 | | 阿拉伯语（ar） | 77,770 | | 日语（ja） | 27,198 | | 他加禄语（tl） | 40,875 | | 西班牙语（es） | 25,609 | | 捷克语（cs） | 15,189 | ## 数据来源 本数据集的种子数据源自多个多语言数据集，随后通过Gemma-3-27B-IT模型生成全新的合成样本。每个源数据集会通过以下三种策略之一进行处理：**generate（基于种子数据生成全新的prompt-response对）**、**respond（针对给定prompt生成响应）**或**translate（将英文样本翻译为目标语言）**。 * [allenai/WildChat-4.8M](https://huggingface.co/datasets/allenai/WildChat-4.8M)：源自真实用户交互的多语言prompt-response对数据集。 * [openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k)：数学应用题数据集（英文版本，已翻译至目标语言）。 * [Magpie-Align/Magpie-Pro-300K-Filtered](https://huggingface.co/datasets/Magpie-Align/Magpie-Pro-300K-Filtered)：通用对话数据集（英文版本，已翻译至目标语言）。 * [nvidia/Helpsteer3](https://huggingface.co/datasets/nvidia/Helpsteer3)：多语言偏好对齐数据集。 * [OpenAssistant/oasst2](https://huggingface.co/datasets/OpenAssistant/oasst2)：多语言助手对话数据集。 * [utter-project/EuroBlocks-SFT-Synthetic-1124](https://huggingface.co/datasets/utter-project/EuroBlocks-SFT-Synthetic-1124)：欧洲多语言合成微调数据集。 * [CohereLabs/aya_collection](https://huggingface.co/datasets/CohereLabs/aya_collection)：多语言指令数据集。 * [arbml/CIDAR](https://huggingface.co/datasets/arbml/CIDAR)：阿拉伯语指令数据集。 * [indonlp/cendol_collection_v2](https://huggingface.co/datasets/indonlp/cendol_collection_v2)：印尼语指令数据集。 ## 数据集结构 每条样本包含以下字段： | 字段名 | 数据类型 | 描述 | |------------|----------|------| | `id` | 字符串 | 唯一标识符 | | `source` | 字符串 | 源数据集名称 | | `language` | 字符串 | ISO 639-1 语言代码 | | `strategy` | 字符串 | 合成策略类型，可选值为`generate`、`respond`或`translate` | | `source_id` | 字符串 | 源数据集中的原始标识符 | | `synth_prompt` | 字符串 | 合成阶段用于指示教师模型的prompt | | `model` | 字符串 | 用于生成样本的教师模型 | | `prompt` | 字符串 | 用户侧prompt | | `response` | 字符串 | 模型生成的响应 | | `messages` | 列表 | 适配监督微调的对话格式消息列表，包含`role`（角色）和`content`（内容）字段 | ## 使用方法 python from datasets import load_dataset ds = load_dataset("ljvmiranda921/PolyglotTeachers-SFT-Synth", split="train") # 按语言筛选样本 arabic_ds = ds.filter(lambda x: x["language"] == "ar") # 直接使用messages字段进行监督微调 print(arabic_ds[0]["messages"]) ## 致谢 LJVM与AK感谢英国研究与创新署（UKRI）前沿项目EP/Y031350/1（[EQUATE](https://gtr.ukri.org/projects?ref=EP%2FY031350%2F1)）的资助。本研究使用了剑桥数据驱动发现服务（Cambridge Service for Data Driven Discovery, CSD3）EP/T022159/1以及Isambard AI国家人工智能研究资源（Isambard AI National AI Research Resource, AIRR）ST/AIRR/I-A-I/1023的联合计算资源，并获得微软研究院研究基金支持。LJVM在此感谢Songbo Hu、Chen Cecilia Liu、Millicent Ochieng与Felermino Ali为本项目提供的富有启发性的讨论。 ## 引用格式 bibtex @misc{miranda2026polyglotteachersevaluatinglanguage, title={Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation}, author={Lester James V. Miranda and Ivan Vulić and Anna Korhonen}, year={2026}, eprint={2604.11290}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2604.11290}, }