SEACrowd/sap_wat

Sap Wat数据集来源于SAP帮助门户，包含SAP产品的文档和产品相关问题的用户帮助。数据集经过处理，适合用于机器翻译的开发和测试。当前的语言范围包括英语到印地语、印尼语、日语、韩语、马来语、泰语、越南语、简体中文和繁体中文。每种语言对大约有4k个段落，分为开发和测试数据。段落以文档上下文提供，并带有文档的额外元数据注释。数据集支持机器翻译任务。

The Sap Wat dataset originates from the SAP Help Portal that contains documentation for SAP products and user assistance for product-related questions. The data has been processed in a way that makes it suitable as development and test data for machine translation purposes. The current language scope is English to Hindi, Indonesian, Japanese, Korean, Malay, Thai, Vietnamese, Simplified Chinese and Traditional Chinese. For each language pair about 4k segments are available, split into development and test data. The segments are provided in their document context and are annotated with additional metadata from the document. The dataset supports machine translation tasks.

原始信息汇总

Sap Wat 数据集概述

数据集来源

• 数据集源自SAP Help Portal，包含SAP产品的文档和用户辅助信息。

数据处理

• 数据经过处理，适合作为机器翻译的开发和测试数据。

语言范围

• 支持的语言对包括：
  • 英语到印地语
  • 英语到印度尼西亚语
  • 英语到日语
  • 英语到韩语
  • 英语到马来语
  • 英语到泰语
  • 英语到越南语
  • 英语到简体中文
  • 英语到繁体中文

数据量

• 每个语言对大约有4000个片段，分为开发和测试数据。

数据格式

• 片段提供在文档上下文中，并带有文档的附加元数据。

支持任务

• 机器翻译

数据集版本

• 源版本：1.0.0
• SEACrowd版本：2024.06.20

许可证

• Creative Commons Attribution Non Commercial 4.0 (cc-by-nc-4.0)

引用

• 使用数据集时，请引用以下文献：

  @inproceedings{buschbeck-exel-2020-parallel, title = "A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation", author = "Buschbeck, Bianka and Exel, Miriam", editor = "Nakazawa, Toshiaki and Nakayama, Hideki and Ding, Chenchen and Dabre, Raj and Kunchukuttan, Anoop and Pa, Win Pa and Bojar, Ond{{r}}ej and Parida, Shantipriya and Goto, Isao and Mino, Hidaya and Manabe, Hiroshi and Sudoh, Katsuhito and Kurohashi, Sadao and Bhattacharyya, Pushpak", booktitle = "Proceedings of the 7th Workshop on Asian Translation", month = dec, year = "2020", address = "Suzhou, China", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.wat-1.20", pages = "160--169", abstract = "This paper accompanies the software documentation data set for machine translation, a parallel evaluation data set of data originating from the SAP Help Portal, that we released to the machine translation community for research purposes. It offers the possibility to tune and evaluate machine translation systems in the domain of corporate software documentation and contributes to the availability of a wider range of evaluation scenarios. The data set comprises of the language pairs English to Hindi, Indonesian, Malay and Thai, and thus also increases the test coverage for the many low-resource language pairs. Unlike most evaluation data sets that consist of plain parallel text, the segments in this data set come with additional metadata that describes structural information of the document context. We provide insights into the origin and creation, the particularities and characteristics of the data set as well as machine translation results.", }

  @article{lovenia2024seacrowd, title={SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages}, author={Holy Lovenia and Rahmad Mahendra and Salsabil Maulana Akbar and Lester James V. Miranda and Jennifer Santoso and Elyanah Aco and Akhdan Fadhilah and Jonibek Mansurov and Joseph Marvin Imperial and Onno P. Kampman and Joel Ruben Antony Moniz and Muhammad Ravi Shulthan Habibi and Frederikus Hudi and Railey Montalan and Ryan Ignatius and Joanito Agili Lopo and William Nixon and Börje F. Karlsson and James Jaya and Ryandito Diandaru and Yuze Gao and Patrick Amadeus and Bin Wang and Jan Christian Blaise Cruz and Chenxi Whitehouse and Ivan Halim Parmonangan and Maria Khelli and Wenyu Zhang and Lucky Susanto and Reynard Adha Ryanda and Sonny Lazuardi Hermawan and Dan John Velasco and Muhammad Dehan Al Kautsar and Willy Fitra Hendria and Yasmin Moslem and Noah Flynn and Muhammad Farid Adilazuarda and Haochen Li and Johanes Lee and R. Damanhuri and Shuo Sun and Muhammad Reza Qorib and Amirbek Djanibekov and Wei Qi Leong and Quyet V. Do and Niklas Muennighoff and Tanrada Pansuwan and Ilham Firdausi Putra and Yan Xu and Ngee Chia Tai and Ayu Purwarianti and Sebastian Ruder and William Tjhi and Peerat Limkonchotiwat and Alham Fikri Aji and Sedrick Keh and Genta Indra Winata and Ruochen Zhang and Fajri Koto and Zheng-Xin Yong and Samuel Cahyawijaya}, year={2024}, eprint={2406.10118}, journal={arXiv preprint arXiv: 2406.10118} }