cturan/turkish-synthetic-corpus

--- license: odc-by task_categories: - text-generation language: - tr size_categories: - 1M<n<10M pretty_name: Turkish Synthetic Corpus --- # Turkish Synthetic Corpus A synthetic Turkish text corpus with **1,871,131** documents, designed for Turkish language model training. ## About Inspired by [HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus). Questions and prompts were sourced from the SmolLM Corpus pipeline; a language model then generated localized Turkish responses and documents around them. All credit for the original corpus design and methodology goes to the HuggingFace SmolLM team. The resulting dataset covers a wide range of topics — science, history, culture, economics, coding, fiction — written in natural Turkish prose at varying register levels. ## Data Fields | Field | Type | Description | |---|---|---| | `id` | `string` | Unique document ID | | `text` | `string` | Turkish document text | ## Intended Use - Turkish LLM pretraining and continued pretraining - Tokenizer training - Language modeling benchmarks > Content is synthetic. Factual accuracy is not guaranteed. ## Citation If you use this dataset, please also credit the original SmolLM Corpus: ```bibtex @software{benallal2024smollmcorpus, author = {Ben Allal, Loubna and Lozhkov, Anton and Penedo, Guilherme and Wolf, Thomas and von Werra, Leandro}, title = {SmolLM-Corpus}, month = July, year = 2024, url = {https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus} } ```license: ODC-BY task_categories: - 文本生成 language: - 土耳其语 size_categories: - 100万<样本数<1000万 pretty_name: 土耳其合成语料库 # 土耳其合成语料库 本数据集为包含1,871,131份文档的土耳其语合成文本语料库，专为土耳其语大语言模型（LLM）训练打造。 ## 数据集说明 本数据集的设计灵感源自[HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus)。其中的问题与提示词均取自SmolLM语料库处理流程；随后借助大语言模型围绕这些内容生成本土化的土耳其语回复与文档。本数据集的原始语料库设计与方法论的全部贡献均归属于HuggingFace SmolLM团队。 最终生成的数据集涵盖科学、历史、文化、经济学、编程、小说等广泛主题，采用不同语体风格的自然土耳其语散文撰写。 ## 数据字段 | 字段 | 类型 | 描述 | |---|---|---| | `id` | 字符串 | 唯一文档标识符 | | `text` | 字符串 | 土耳其语文本文档内容 | ## 预期用途 - 土耳其语大语言模型预训练与持续预训练 - 分词器（Tokenizer）训练 - 语言模型基准测试 > 本数据集内容为合成生成，不保证事实准确性。 ## 引用说明 若使用本数据集，请同时注明原始SmolLM语料库的来源： bibtex @software{benallal2024smollmcorpus, author = {Ben Allal, Loubna and Lozhkov, Anton and Penedo, Guilherme and Wolf, Thomas and von Werra, Leandro}, title = {SmolLM-Corpus}, month = July, year = 2024, url = {https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus} }