hezarai/parstwiner

---
language:
- fa
task_categories:
- token-classification
pretty_name: ParsTwiNER
dataset_info:
features:
- name: tokens
sequence: string
- name: ner_tags
sequence:
class_label:
names:
'0': O
'1': B-POG
'2': I-POG
'3': B-PER
'4': I-PER
'5': B-ORG
'6': I-ORG
'7': B-NAT
'8': I-NAT
'9': B-LOC
'10': I-LOC
'11': B-EVE
'12': I-EVE
splits:
- name: train
num_bytes: 4434479
num_examples: 6865
- name: test
num_bytes: 198933
num_examples: 304
download_size: 1041183
dataset_size: 4633412
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
---
ParsTwiNER dataset created by Aghajani et al. [Paper](https://paperswithcode.com/paper/parstwiner-a-corpus-for-named-entity)

> As a result of unstructured sentences and some misspellings and errors, finding named entities in a noisy environment such as social media takes much more effort.
> ParsTwiNER contains about 250k tokens, based on standard instructions like MUC-6 or CoNLL 2003, gathered from Persian Twitter. Using Cohen’s Kappa coefficient, the consistency of annotators is 0.95, a high score.
> In this study, we demonstrate that some state-of-the-art models degrade on these corpora, and trained a new model using parallel transfer learning based on the BERT architecture.
> Experimental results show that the model works well in informal Persian as well as in formal Persian.

原始信息汇总

ParsTwiNER 数据集概述

数据集基本信息

• 语言: 波斯语 (fa)
• 任务类别: 词元分类
• 数据集名称: ParsTwiNER

数据集特征

• tokens: 字符串序列
• ner_tags: 命名实体标签序列，包含以下类别：
  • 0: O
  • 1: B-POG
  • 2: I-POG
  • 3: B-PER
  • 4: I-PER
  • 5: B-ORG
  • 6: I-ORG
  • 7: B-NAT
  • 8: I-NAT
  • 9: B-LOC
  • 10: I-LOC
  • 11: B-EVE
  • 12: I-EVE

数据集分割

• 训练集: 6865个样本，大小为4434479字节
• 测试集: 304个样本，大小为198933字节

数据集大小

• 下载大小: 1041183字节
• 数据集总大小: 4633412字节

数据文件配置

• 默认配置:
  • 训练集路径: data/train-*
  • 测试集路径: data/test-*