kurdish-ai/kurdish-corpus

库尔德语语料库是一个大规模、多来源的库尔德语数据集，专门用于训练语言模型。该数据集覆盖四种库尔德语方言：Sorani（ckb）、Kurmanji（kmr）、Zazaki（diq）和Hawrami（hac），包含总计1,797,686个文档和625,716,980个词元。数据来源于新闻、网页和维基百科等多种渠道，其中新闻类文档占比最高。数据集提供了丰富的元数据，如语言代码、来源类型、质量评分（0-100）、词数统计和URL等，支持文本生成和掩码填充等NLP任务。数据集采用CC-BY-4.0许可证，于2026年5月2日构建，旨在促进库尔德语自然语言处理研究和应用。

A large-scale multi-source Kurdish language dataset for training language models. It covers four Kurdish dialects: Sorani (ckb), Kurmanji (kmr), Zazaki (diq), and Hawrami (hac), with a total of 1,797,686 documents and 625,716,980 tokens. The data is sourced from multiple channels including news, web, and Wikipedia, with news documents being the most prevalent. The dataset includes rich metadata such as language codes, source types, quality scores (0-100), word counts, and URLs, supporting NLP tasks like text generation and fill-mask. Licensed under CC-BY-4.0, it was built on May 2, 2026, and aims to advance Kurdish language natural language processing research and applications.