hezarai/lscp-pos-500k

--- language: - fa task_categories: - token-classification pretty_name: LSCP Dataset (500k samples version) dataset_info: features: - name: tokens sequence: string - name: pos_tags sequence: class_label: names: '0': P '1': Ne '2': PRO '3': CONJ '4': N '5': PUNC '6': Pe '7': ADV '8': V '9': AJ '10': AJe '11': DET '12': POSTP '13': NUM '14': DETe '15': NUMe '16': PROe '17': ADVe '18': RES '19': CL '20': INT '21': CONJe '22': RESe splits: - name: train num_bytes: 156464559 num_examples: 449998 - name: test num_bytes: 17384986 num_examples: 50000 download_size: 46556766 dataset_size: 173849545 configs: - config_name: default data_files: - split: train path: data/train-* - split: test path: data/test-* --- This is a 500 thousand sample version of the original [LSCP dataset](https://iasbs.ac.ir/~ansari/lscp/) that only contains the text and part-of-speech tags and is used for sequence labeling. ### Citation ```bibtex @InProceedings{abdikhojasteh:2020:LREC, author = {Abdi Khojasteh, Hadi and Ansari, Ebrahim and Bohlouli, Mahdi}, title = {LSCP: Enhanced Large Scale Colloquial Persian Language Understanding}, booktitle = {Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020)}, year = {2020} address = {Marseille, France}, publisher = {European Language Resources Association} pages = {6323--6327}, url = {https://www.aclweb.org/anthology/2020.lrec-1.776} } ```

原始信息汇总

数据集概述

数据集名称

• 名称: LSCP Dataset (500k samples version)

语言

• 语言: 波斯语 (fa)

任务类别

• 任务类别: 词元分类 (token-classification)

数据集特征

• tokens: 字符串序列
• pos_tags: 序列标签，包括以下类别：
  • P, Ne, PRO, CONJ, N, PUNC, Pe, ADV, V, AJ, AJe, DET, POSTP, NUM, DETe, NUMe, PROe, ADVe, RES, CL, INT, CONJe, RESe

数据集拆分

• 训练集:
  • 样本数: 449998
  • 数据大小: 156464559 字节
• 测试集:
  • 样本数: 50000
  • 数据大小: 17384986 字节

数据集大小

• 下载大小: 46556766 字节
• 总数据大小: 173849545 字节

配置文件

• 默认配置:
  • 训练数据路径: data/train-*
  • 测试数据路径: data/test-*