BOLT Egyptian Arabic SMS/Chat and Transliteration

Introduction BOLT Egyptian Arabic SMS/Chat and Transliteration was developed by the Linguistic Data Consortium (LDC) and consists of naturally-occurring Short Message Service (SMS) and Chat (CHT) data collected through data donations and live collection involving native speakers of Egyptian Arabic. The corpus contains 5,691 conversations totaling 1,029,248 words across 262,026 messages. Messages were natively written in either Arabic orthography or romanized Arabizi. A total of 1,856 Arabizi conversations (287,022 words) were transliterated from the original romanized Arabizi script into standard Arabic orthography. The BOLT (Broad Operational Language Translation) program developed machine translation and information retrieval for less formal genres, focusing particularly on user-generated content. LDC supported the BOLT program by collecting informal data sources -- discussion forums, text messaging and chat -- in Chinese, Egyptian Arabic and English. The collected data was translated and annotated for various tasks including word alignment, treebanking, propbanking and co-reference. Data The data in this release was collected using two methods: new collection via LDC's collection platform, and donation of SMS or chat archives from BOLT collection participants. All data collected were reviewed manually to exclude any messages/conversations that were not in the target language or that had sensitive content, such as personal identifying information (PII). A portion of the source conversations containing Arabizi tokens were automatically transliterated into Arabic script. Once the Arabizi source was transliterated into Arabic script automatically, LDC annotators reviewed, corrected and normalized the transliteration according to "Conventional Orthography for Dialectal Arabic" (CODA). All data is presented in XML. Acknowledgement This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-11-C-0145. The content does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. Samples Please view this conversation sample and transliteration sample. Updates None at this time. Portions © 2017 Trustees of the University of Pennsylvania

简介
BOLT埃及阿拉伯语短信/聊天与音译数据集由语言数据联盟（Linguistic Data Consortium，LDC）开发，其数据源自面向埃及阿拉伯语母语者的捐赠采集与现场采集得到的自然产生的短消息服务（SMS）与聊天（CHT）文本。该语料库包含5691段对话，共计262026条消息，总词数达1029248词。消息原文采用阿拉伯语正字法或罗马化阿拉伯语（Arabizi）书写。其中1856段阿拉伯语罗马化对话（共计287022词）已从原罗马化阿拉伯语文本音译为标准阿拉伯语正字法。

BOLT（Broad Operational Language Translation，通用作战语言翻译）项目旨在为非规范文体开发机器翻译与信息检索技术，尤其聚焦于用户生成内容。LDC通过采集中文、埃及阿拉伯语与英语的非正式数据源——包括论坛、短信与聊天文本——为该项目提供支持。所采集的数据已针对词对齐、树库标注、论元结构标注（propbanking）与共指消解等任务完成翻译与标注。

数据
本次发布的数据采用两种采集方式：通过LDC官方采集平台进行的全新采集，以及来自BOLT项目参与者捐赠的短信或聊天档案。所有采集到的数据均经过人工审核，以排除非目标语言文本或包含敏感内容（如个人身份信息（PII））的消息与对话。部分包含阿拉伯语罗马化Token的原始对话已被自动音译为阿拉伯语书写系统。在自动完成阿拉伯语罗马化文本到阿拉伯语正字法的音译后，LDC的标注人员会依据《方言阿拉伯语规范正字法》（Conventional Orthography for Dialectal Arabic，CODA）对音译结果进行审核、修正与规范化处理。所有数据均以可扩展标记语言（XML）格式呈现。

致谢
本材料基于美国国防高级研究计划局（Defense Advanced Research Projects Agency，DARPA）合同号HR0011-11-C-0145资助的研究工作。本内容不一定代表政府的立场或政策，不应推断存在任何官方背书。

样本
请查看本次对话样本与音译样本。

更新
暂无最新更新。

部分内容 © 2017 宾夕法尼亚大学托管委员会

搜集汇总

背景与挑战

背景概述

BOLT Egyptian Arabic SMS/Chat and Transliteration是一个埃及阿拉伯语短信和聊天数据集，包含5,691个对话和1,029,248个单词，数据以阿拉伯文字或罗马化的Arabizi书写，部分Arabizi对话被音译为标准阿拉伯文字。该数据集由Linguistic Data Consortium开发，属于BOLT项目，旨在支持机器翻译和信息检索应用，专注于非正式文本类型，如用户生成内容。