Adyan: A High-Quality Automated NER Dataset for Sorani Kurdish

The Named Entity Recognition Dataset for Kurdish Sorani (Adian) is an automated annotated resource created to support NLP studies for Kurdish Sorani, a low-resource language. By providing a large, structured dataset that trains and evaluates Kurdish language models, it addresses the lack of annotated corpora for nominal entity identification (NER).

The dataset of more than 2,300 news articles published in 2024 by trusted Kurdish media sources such as Rudaw, NRT, and Channel 8 covers six main domains: politics, economy, sports, culture, interviews, and technology. It contains 641,871 tokens across 24,820 strings. A predefined dictionary of 15,872 named entities was used for annotation, covering 15 entity types. Annotation was performed automatically using a dictionary-based approach and the BIO tagging scheme. Preprocessing steps such as text cleaning, normalization, and duplicate removal were implemented using Python scripts to improve consistency and quality.

Adyan is publicly available for academic and research use. Its scale, domain coverage and automated methodology make it a valuable resource not only for NER but also for sentiment analysis, tool translation and other Kurdish NLP tasks. The dataset makes a significant contribution to the advancement of language technology for underrepresented languages.

库尔德索拉尼语命名实体识别（Named Entity Recognition, NER）数据集（Adian）是一款自动化标注资源，旨在为低资源语言库尔德索拉尼语的自然语言处理（Natural Language Processing, NLP）研究提供支持。通过提供大规模结构化数据集以训练与评估库尔德语语言模型，该数据集填补了库尔德语命名实体识别领域标注语料库匮乏的空白。

该数据集收录了2024年由鲁达瓦（Rudaw）、NRT、第八频道（Channel 8）等权威库尔德媒体发布的2300余篇新闻文章，涵盖政治、经济、体育、文化、访谈、科技六大核心领域。数据集共包含24820个字符串单元，总计641871个Token。标注环节采用预定义的15872个命名实体词典，覆盖15种实体类型，并通过基于词典的自动化方法与BIO标注体系完成。为提升数据一致性与质量，研究团队通过Python脚本实现了文本清洗、归一化及去重等预处理步骤。

Adyan面向学术与研究用途公开开放。其规模、领域覆盖范围与自动化标注方法，使其不仅可用于命名实体识别任务，还可应用于情感分析、工具翻译及其他库尔德语自然语言处理任务。该数据集为弱势语言的语言技术发展作出了重要贡献。