uznlp-uz/uz_medner

--- license: cc-by-4.0 language: - uz pretty_name: UZ-MedNER v1.0 size_categories: - 1K<n<10K task_categories: - text-classification configs: - config_name: default default: true data_files: - split: train path: UzMedNER.tsv sep: "\t" - config_name: tagset data_files: - split: train path: tagset.tsv sep: "\t" --- # Uzbek Medical NER Dataset (UzMedNER) ## 📌 Description This dataset introduces **UzMedNER**, a structured Named Entity Recognition (NER) resource for the Uzbek language in the **medical domain**. It is designed to support token-level sequence labeling tasks and facilitate research in low-resource biomedical NLP. The dataset consists of manually annotated Uzbek text where each token is labeled using a predefined tagset representing medical and related entity types. UzMedNER addresses the lack of: * domain-specific annotated corpora in Uzbek * standardized NER benchmarks for medical text * resources for training sequence labeling models in low-resource settings --- ## 🧠 Task Definition This dataset is designed for: ### Named Entity Recognition (NER) * **Input:** tokenized Uzbek sentence * **Output:** sequence of entity labels (BIO tagging scheme) Example: ```text Bemor B-DISEASE diabet I-DISEASE bilan O kasallangan O . ``` --- ## 📊 Dataset Structure The dataset is stored in **TSV format** with token-level annotations. Typical format: ```tsv token label Bemor O diabet B-DISEASE bilan O kasallangan O ``` * Each row = one token * Labels follow **BIO tagging scheme** * Sentences are separated by empty lines --- ## 🏷 Tagset (Entity Types) The dataset uses a BIO-based tagging scheme with the following entity categories: | Tag | Description | | ------------------------- | -------------------------- | | B-DISEASE / I-DISEASE | Disease names | | B-SYMPTOM / I-SYMPTOM | Symptoms | | B-DRUG / I-DRUG | Medications | | B-TREATMENT / I-TREATMENT | Medical treatments | | B-ANATOMY / I-ANATOMY | Body parts | | B-TEST / I-TEST | Medical tests | | O | Outside (non-entity token) | > Note: Exact tag inventory is defined in the accompanying `tagset.tsv` file. --- ## 🧾 Example ```text Token Label Bemor O yurak B-ANATOMY og‘rig‘i B-SYMPTOM bilan O shifoxonaga O murojaat O qildi O ``` --- ## 📏 Evaluation Protocol Recommended evaluation metrics: * Precision * Recall * F1-score (entity-level) * Token-level accuracy Evaluation should follow standard **CoNLL NER evaluation**. --- ## 📊 Data Splits *Note: predefined splits may be added in future versions.* Recommended split: * Train: 80% * Validation: 10% * Test: 10% --- ## 🎯 Use Cases This dataset can be used for: * 🏥 Medical NER in Uzbek * 🤖 Fine-tuning transformer models (BERT, RoBERTa, Qwen, etc.) * 📊 Sequence labeling research * 🔍 Clinical text mining * 🧠 Biomedical NLP for low-resource languages --- ## ⚙️ Loading the Dataset ```python from datasets import load_dataset dataset = load_dataset("ruhilloalaev/UzMedNER", "default") ``` --- ## ⚠️ Notes * Data is in **Uzbek (Latin script)** * Annotation follows **BIO scheme** * Domain: **medical / clinical language** * Some entities may exhibit: * morphological variation * spelling inconsistencies * domain-specific abbreviations --- ## 📜 License This dataset is released under the **CC-BY-4.0 License**.许可协议：CC BY 4.0（CC-BY-4.0） 语言：乌兹别克语 正式命名：UZ-MedNER v1.0 数据规模分类：1000 < 样本数量 < 10000 任务分类：文本分类 配置项： 1. 默认配置（default）：设为默认配置，数据文件为`UzMedNER.tsv`，对应训练拆分，分隔符为制表符（ ） 2. 标签集配置（tagset）：数据文件为`tagset.tsv`，对应训练拆分，分隔符为制表符（ ） # 乌兹别克语医疗命名实体识别数据集（UzMedNER） ## 📌 数据集概述 本数据集推出**UzMedNER**，一款面向乌兹别克语医疗领域的结构化命名实体识别（Named Entity Recognition, NER）资源，旨在支撑令牌级序列标注任务，助力低资源生物医学自然语言处理领域的研究。 本数据集由经过人工标注的乌兹别克语文本构成，每一个令牌均通过预定义的标签集进行标注，该标签集涵盖医疗及相关实体类型。 UzMedNER填补了以下领域的资源空白： - 乌兹别克语领域专属标注语料库的缺失 - 医疗文本标准化NER评测基准的缺失 - 低资源场景下序列标注模型训练所需资源的缺失 ## 🧠 任务定义 本数据集适配以下任务： ### 命名实体识别（NER） - **输入**：经过分词的乌兹别克语语句 - **输出**：实体标签序列（采用BIO标注体系（BIO tagging scheme）） 示例： text Bemor B-DISEASE diabet I-DISEASE bilan O kasallangan O . ## 📊 数据集结构 本数据集采用**TSV（制表符分隔值）格式**存储，包含令牌级标注信息。 典型格式如下： tsv token label Bemor O diabet B-DISEASE bilan O kasallangan O - 每一行对应一个令牌 - 标签遵循**BIO标注体系** - 语句之间以空行分隔 ## 🏷 标签集（实体类型） 本数据集采用基于BIO的标注体系，实体类别如下： | 标签 | 描述 | | ------------------------ | ------------------------ | | B-DISEASE / I-DISEASE | 疾病名称 | | B-SYMPTOM / I-SYMPTOM | 症状 | | B-DRUG / I-DRUG | 药物 | | B-TREATMENT / I-TREATMENT| 医疗干预手段 | | B-ANATOMY / I-ANATOMY | 身体部位 | | B-TEST / I-TEST | 医学检查 | | O | 非实体令牌（Outside） | > 注：完整标签清单详见配套的`tagset.tsv`文件。 ## 🧾 标注示例 text Token Label Bemor O yurak B-ANATOMY og‘rig‘i B-SYMPTOM bilan O shifoxonaga O murojaat O qildi O ## 📏 评测协议 推荐采用以下评测指标： - 精确率（Precision） - 召回率（Recall） - 实体级F1值（F1-score） - 令牌级准确率 评测需遵循标准的**CoNLL NER评测规范（CoNLL NER evaluation）**。 ## 📊 数据拆分 *注：未来版本可能会新增预定义的数据拆分。* 推荐拆分比例为： - 训练集：80% - 验证集：10% - 测试集：10% ## 🎯 应用场景 本数据集可应用于以下场景： - 🏥 乌兹别克语医疗领域NER任务 - 🤖 Transformer模型（BERT、RoBERTa、Qwen等）的微调 - 📊 序列标注相关研究 - 🔍 临床文本挖掘 - 🧠 低资源语言的生物医学自然语言处理研究 ## ⚙️ 数据集加载 python from datasets import load_dataset dataset = load_dataset("ruhilloalaev/UzMedNER", "default") ## ⚠️ 注意事项 - 数据采用**乌兹别克语拉丁字母书写** - 标注遵循**BIO标注体系** - 领域：**医疗/临床文本** - 部分实体可能存在以下情况： 1. 形态变体 2. 拼写不一致 3. 领域专属缩写 ## 📜 许可协议 本数据集采用**CC BY 4.0（CC-BY-4.0）许可协议**发布。