SEACrowd/idner_news_2k
收藏Idner News 2K 数据集概述
基本信息
- 名称: Idner News 2K
- 语言: 印度尼西亚语 (ind)
- 任务类别: 命名实体识别 (Named Entity Recognition)
- 标签: 命名实体识别
- 许可证: MIT
数据集描述
- 任务: 命名实体识别
- 来源: 该数据集重新标注了由 Syaifudin & Nurwidyantoro (2016) 提供的原始数据集，使用更标准化的 NER 标签。
- 子集: 包含三个子集，分别是
train.txt、dev.txt和test.txt。 - 文件格式: 每个文件包含三列，分别是 Tokens、PoS Tag 和 NER Tag。
- 格式: 遵循 CoNLL 数据集格式，NER 标签使用 IOB 格式。
- PoS 标签: 使用 UDPipe 进行标注，该工具是一个用于分词、词性标注、词形还原和依存句法分析的管道，其模型基于 UD Treebanks 训练。
数据集版本
- 源版本: 1.0.0
- SEACrowd 版本: 2024.06.20
引用
-
引用方式:
@inproceedings{khairunnisa-etal-2020-towards, title = "Towards a Standardized Dataset on {I}ndonesian Named Entity Recognition", author = "Khairunnisa, Siti Oryza and Imankulova, Aizhan and Komachi, Mamoru", editor = "Shmueli, Boaz and Huang, Yin Jou", booktitle = "Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: Student Research Workshop", month = dec, year = "2020", address = "Suzhou, China", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.aacl-srw.10", pages = "64--71", abstract = "In recent years, named entity recognition (NER) tasks in the Indonesian language have undergone extensive development. There are only a few corpora for Indonesian NER; hence, recent Indonesian NER studies have used diverse datasets. Although an open dataset is available, it includes only approximately 2,000 sentences and contains inconsistent annotations, thereby preventing accurate training of NER models without reliance on pre-trained models. Therefore, we re-annotated the dataset and compared the two annotations{} performance using the Bidirectional Long Short-Term Memory and Conditional Random Field (BiLSTM-CRF) approach. Fixing the annotation yielded a more consistent result for the organization tag and improved the prediction score by a large margin. Moreover, to take full advantage of pre-trained models, we compared different feature embeddings to determine their impact on the NER task for the Indonesian language.", }
@article{lovenia2024seacrowd, title={SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages}, author={Holy Lovenia and Rahmad Mahendra and Salsabil Maulana Akbar and Lester James V. Miranda and Jennifer Santoso and Elyanah Aco and Akhdan Fadhilah and Jonibek Mansurov and Joseph Marvin Imperial and Onno P. Kampman and Joel Ruben Antony Moniz and Muhammad Ravi Shulthan Habibi and Frederikus Hudi and Railey Montalan and Ryan Ignatius and Joanito Agili Lopo and William Nixon and Börje F. Karlsson and James Jaya and Ryandito Diandaru and Yuze Gao and Patrick Amadeus and Bin Wang and Jan Christian Blaise Cruz and Chenxi Whitehouse and Ivan Halim Parmonangan and Maria Khelli and Wenyu Zhang and Lucky Susanto and Reynard Adha Ryanda and Sonny Lazuardi Hermawan and Dan John Velasco and Muhammad Dehan Al Kautsar and Willy Fitra Hendria and Yasmin Moslem and Noah Flynn and Muhammad Farid Adilazuarda and Haochen Li and Johanes Lee and R. Damanhuri and Shuo Sun and Muhammad Reza Qorib and Amirbek Djanibekov and Wei Qi Leong and Quyet V. Do and Niklas Muennighoff and Tanrada Pansuwan and Ilham Firdausi Putra and Yan Xu and Ngee Chia Tai and Ayu Purwarianti and Sebastian Ruder and William Tjhi and Peerat Limkonchotiwat and Alham Fikri Aji and Sedrick Keh and Genta Indra Winata and Ruochen Zhang and Fajri Koto and Zheng-Xin Yong and Samuel Cahyawijaya}, year={2024}, eprint={2406.10118}, journal={arXiv preprint arXiv: 2406.10118} }
