hezarai/lscp-pos-500k

---
language:
- fa
task_categories:
- token-classification
pretty_name: LSCP Dataset (500k samples version)
dataset_info:
features:
- name: tokens
sequence: string
- name: pos_tags
sequence:
class_label:
names:
'0': P
'1': Ne
'2': PRO
'3': CONJ
'4': N
'5': PUNC
'6': Pe
'7': ADV
'8': V
'9': AJ
'10': AJe
'11': DET
'12': POSTP
'13': NUM
'14': DETe
'15': NUMe
'16': PROe
'17': ADVe
'18': RES
'19': CL
'20': INT
'21': CONJe
'22': RESe
splits:
- name: train
num_bytes: 156464559
num_examples: 449998
- name: test
num_bytes: 17384986
num_examples: 50000
download_size: 46556766
dataset_size: 173849545
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
---

This is a 500 thousand sample version of the original [LSCP dataset](https://iasbs.ac.ir/~ansari/lscp/) that only contains the text and part-of-speech tags and is used for sequence labeling.

### Citation
```bibtex
@InProceedings{abdikhojasteh:2020:LREC,
author = {Abdi Khojasteh, Hadi and Ansari, Ebrahim and Bohlouli, Mahdi},
title = {LSCP: Enhanced Large Scale Colloquial Persian Language Understanding},
booktitle = {Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020)},
year = {2020}
address = {Marseille, France},
publisher = {European Language Resources Association}
pages = {6323--6327},
url = {https://www.aclweb.org/anthology/2020.lrec-1.776}
}
```

原始信息汇总

数据集概述

数据集名称

• 名称: LSCP Dataset (500k samples version)

语言

• 语言: 波斯语 (fa)

任务类别

• 任务类别: 词元分类 (token-classification)

数据集特征

• tokens: 字符串序列
• pos_tags: 序列标签，包括以下类别：
  • P, Ne, PRO, CONJ, N, PUNC, Pe, ADV, V, AJ, AJe, DET, POSTP, NUM, DETe, NUMe, PROe, ADVe, RES, CL, INT, CONJe, RESe

数据集拆分

• 训练集:
  • 样本数: 449998
  • 数据大小: 156464559 字节
• 测试集:
  • 样本数: 50000
  • 数据大小: 17384986 字节

数据集大小

• 下载大小: 46556766 字节
• 总数据大小: 173849545 字节

配置文件

• 默认配置:
  • 训练数据路径: data/train-*
  • 测试数据路径: data/test-*