GALE Phase 1 Chinese Broadcast Conversation Parallel Text - Part 1

Name: GALE Phase 1 Chinese Broadcast Conversation Parallel Text - Part 1
Creator: Linguistic Data Consortium
Published: 2020-11-30T15:50:38+08:00

DataCite Commons2021-07-01 更新2025-04-16 收录

自然语言处理

机器翻译

数据链接：

https://catalog.ldc.upenn.edu/LDC2009T02 数据链接链接失效反馈

官方服务：

资源简介：

<h3>Introduction:</h3> GALE Phase 1 Chinese Broadcast Conversation Parallel Text - Part 1, Linguistic Data Consortium (LDC) catalog number LDC2009T02 and ISBN 1-58563-499-9, contains transcripts and English translations of 20.4 hours of Chinese broadcast conversation programming from China Central TV (CCTV) and Phoenix TV. It does not contain the audio files from which the transcripts and translations were generated. GALE Phase 1 Chinese Broadcast Conversation Parallel Text - Part 1, along with other corpora, was used as training data in year 1 (Phase 1) of the DARPA-funded GALE program. <h3>Source Data:</h3> A total of 20.4 hours of Chinese broadcast conversation programming were selected from two sources: CCTV (a broadcaster from Mainland China), and Phoenix TV (a Hong Kong -based satellite TV station). The transcripts and translations represent recordings of eight different programs. A manual selection procedure was used to choose data appropriate for the GALE program, namely conversation (talk) programs focusing on current events. Stories on topics such as sports, entertainment and business were excluded from the data set. The following table is a summary of the files included in this release. <table><tbody> <tr> <td width="108"> Source </td> <td width="164"> Program </td> <td width="162"> Epoch (YYYY.MM) </td> <td width="96"> #hours </td> <td width="105"> #characters </td> </tr> <tr> <td rowspan="2" width="108"> CCTV </td> <td width="164"> Across China </td> <td width="162"> 2005.08 </td> <td width="96"> 1.0 </td> <td width="105"> 9,924 </td> </tr> <tr> <td width="164"> Todays Focus </td> <td width="162"> 2005.11 </td> <td width="96"> 2.2 </td> <td width="105"> 33,805 </td> </tr> <tr> <td colspan="1" rowspan="6" width="108"> Phoenix TV </td> <td width="164"> Asian Journal </td> <td width="162"> 2005.09 </td> <td width="96"> 2.2 </td> <td width="105"> 26,656 </td> </tr> <tr> <td width="164"> Behind the Headlines </td> <td width="162"> 2005.03 - 2005.11 </td> <td width="96"> 1.5 </td> <td width="105"> 17,933 </td> </tr> <tr> <td> A Date With Lu Yu </td> <td> 2005.09 - 2005.10 </td> <td> 7.1 </td> <td> 89,987 </td> </tr> <tr> <td> News Hacker </td> <td> 2005.03 - 2005.10 </td> <td> 2.3 </td> <td> 39,388 </td> </tr> <tr> <td> Newsline </td> <td> 2005.10 - 2005.11 </td> <td> 1.6 </td> <td> 15,496 </td> </tr> <tr> <td> Social Watch </td> <td> 2005.09 - 2005.11 </td> <td> 2.5 </td> <td> 29,159 </td> </tr> </tbody></table><h3>Transcription:</h3> The selected audio snippets were carefully transcribed by LDC annotators and professional transcription agencies following LDCs Quick Rich Transcription specification. Manual sentence units/segments (SU) annotation was also performed as part of the transcription task. Three types of end of sentence SU are identified: <ul> <li> statement SU </li> <li> question SU </li> <li> incomplete SU </li> </ul><h3>Translation:</h3> After transcription and SU annotation, files were reformatted into a human-readable translation format and assigned to professional translators for careful translation. Translators followed LDCs GALE Translation guidelines which describe the makeup of the translation team, the source data format, the translation data format, best practices for translating certain linguistic features (such as names and speech disfluencies) and quality control procedures applied to completed translations. <h3>TDF Format:</h3> All final data are in Tab Delimited Format (TDF). TDF is compatible with other transcription formats, such as the Transcriber format and AG format, and it is easy to process. Each line of a TDF file corresponds to a speech segment and contains 13 tab delimited fields: <table width="259"><tbody> <tr> <td> Field </td> <td> Data Type </td> </tr> <tr> <td> file </td> <td>unicode </td> </tr> <tr> <td>channel </td> <td>int </td> </tr> <tr> <td>start </td> <td>float </td> </tr> <tr> <td>end </td> <td>float </td> </tr> <tr> <td>speaker </td> <td>unicode </td> </tr> <tr> <td>speakerType </td> <td>unicode </td> </tr> <tr> <td> speakerDialect </td> <td>unicode </td> </tr> <tr> <td>transcript </td> <td>unicode </td> </tr> <tr> <td>section </td> <td>int </td> </tr> <tr> <td>turn </td> <td>int </td> </tr> <tr> <td>segment </td> <td>int </td> </tr> <tr> <td>sectionType </td> <td>unicode </td> </tr> <tr> <td>suType </td> <td>unicode </td> </tr> </tbody></table> A source TDF file and its translation are the same except that the transcript in the source TDF is replaced by its English translation. <h3>Sponsorship</h3> This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. <h3>Samples</h3> For an example of the data in this corpus, please examine these images of the <a href="./desc/addenda/LDC2009T02_source.png" rel="nofollow">source</a> and<a href="./desc/addenda/LDC2009T02_trans.png" rel="nofollow"> translation</a>. Portions © 2005 China Central TV, © 2005 Phoenix TV, © 2005 - 2007, 2009 Trustees of the University of Pennsylvania.

<h3>简介：</h3> GALE第一阶段中文广播对话平行语料库（第一部分）由语言数据联盟（Linguistic Data Consortium, LDC）发布，目录号为LDC2009T02，ISBN为1-58563-499-9。该语料库包含来自中国中央电视台（CCTV）与凤凰卫视的20.4小时中文广播对话节目的转写文本及英文译文，未附带生成转写与译文的原始音频文件。本语料库与其他语料一同作为美国国防高级研究计划局（Defense Advanced Research Projects Agency, DARPA）资助的GALE项目第一年度（第一阶段）的训练数据使用。 <h3>源数据：</h3> 本次数据集共收录20.4小时的中文广播对话节目，素材来源于两大渠道：中国大陆的中国中央电视台（CCTV），以及总部位于香港的卫星电视台凤凰卫视。所有转写文本与译文对应8档不同节目。 本次数据采用人工遴选流程，筛选适配GALE项目需求的内容，即聚焦时事的谈话类对话节目。体育、娱乐与商业类主题的内容均被排除在数据集之外。下表为本批次发布文件的汇总信息。 <table><tbody> <tr> <td width="108"> 来源 </td> <td width="164"> 节目名称 </td> <td width="162"> 录制时段（YYYY.MM） </td> <td width="96"> 时长（小时） </td> <td width="105"> 字符数 </td> </tr> <tr> <td rowspan="2" width="108"> CCTV </td> <td width="164"> 走遍中国（Across China） </td> <td width="162"> 2005.08 </td> <td width="96"> 1.0 </td> <td width="105"> 9,924 </td> </tr> <tr> <td width="164"> 今日关注（Todays Focus） </td> <td width="162"> 2005.11 </td> <td width="96"> 2.2 </td> <td width="105"> 33,805 </td> </tr> <tr> <td colspan="1" rowspan="6" width="108"> 凤凰卫视（Phoenix TV） </td> <td width="164"> 亚洲新闻周刊（Asian Journal） </td> <td width="162"> 2005.09 </td> <td width="96"> 2.2 </td> <td width="105"> 26,656 </td> </tr> <tr> <td width="164"> 新闻背后（Behind the Headlines） </td> <td width="162"> 2005.03 - 2005.11 </td> <td width="96"> 1.5 </td> <td width="105"> 17,933 </td> </tr> <tr> <td> 鲁豫有约（A Date With Lu Yu） </td> <td> 2005.09 - 2005.10 </td> <td> 7.1 </td> <td> 89,987 </td> </tr> <tr> <td> 新闻黑客（News Hacker） </td> <td> 2005.03 - 2005.10 </td> <td> 2.3 </td> <td> 39,388 </td> </tr> <tr> <td> 新闻线（Newsline） </td> <td> 2005.10 - 2005.11 </td> <td> 1.6 </td> <td> 15,496 </td> </tr> <tr> <td> 社会观察（Social Watch） </td> <td> 2005.09 - 2005.11 </td> <td> 2.5 </td> <td> 29,159 </td> </tr> </tbody></table><h3>转写流程：</h3> 遴选后的音频片段由LDC标注人员及专业转写机构遵照LDC快速富转录（Quick Rich Transcription）规范完成转写。转写任务同时包含人工句子单元（Sentence Unit, SU）标注环节，共识别三类句子单元结尾类型： <ul> <li> 陈述型句子单元 </li> <li> 疑问型句子单元 </li> <li> 不完整型句子单元 </li> </ul><h3>译文制作：</h3> 完成转写与句子单元标注后，文件将被重新格式化为易读的译文格式，并交由专业译员进行精准翻译。译员需遵循LDC发布的GALE翻译指南，该指南涵盖翻译团队构成、源数据格式、译文数据格式、特定语言特征（如人名与言语不流畅现象）的翻译规范，以及译文完成后的质量管控流程。 <h3>制表符分隔格式（Tab Delimited Format, TDF）：</h3> 所有最终数据均采用制表符分隔格式（TDF）存储。该格式兼容Transcriber格式与AG格式等其他转录格式，且易于处理。 每个TDF文件的行对应一条语音片段，共包含13个制表符分隔的字段： <table width="259"><tbody> <tr> <td> 字段 </td> <td> 数据类型 </td> </tr> <tr> <td> file </td> <td>unicode </td> </tr> <tr> <td> channel </td> <td>整数型 </td> </tr> <tr> <td> start </td> <td>浮点型（起始时间） </td> </tr> <tr> <td> end </td> <td>浮点型（结束时间） </td> </tr> <tr> <td> speaker </td> <td>unicode（发言者标识） </td> </tr> <tr> <td> speakerType </td> <td>unicode（发言者类型） </td> </tr> <tr> <td> speakerDialect </td> <td>unicode（发言者方言） </td> </tr> <tr> <td> transcript </td> <td>unicode（转写文本） </td> </tr> <tr> <td> section </td> <td>整数型（章节编号） </td> </tr> <tr> <td> turn </td> <td>整数型（轮次编号） </td> </tr> <tr> <td> segment </td> <td>整数型（片段编号） </td> </tr> <tr> <td> sectionType </td> <td>unicode（章节类型） </td> </tr> <tr> <td> suType </td> <td>unicode（句子单元类型） </td> </tr> </tbody></table> 源语言TDF文件与译文TDF文件的结构完全一致，仅将源语言TDF中的转写文本替换为对应的英文译文。 <h3>资助说明</h3> 本工作部分由美国国防高级研究计划局（Defense Advanced Research Projects Agency, DARPA）的GALE项目资助，资助编号为HR0011-06-1-0003。本出版物内容未必代表美国政府的立场或政策，不应被视为获得官方背书。 <h3>样本</h3> 如需查看本语料库的数据示例，请参阅以下链接中的<a href="./desc/addenda/LDC2009T02_source.png" rel="nofollow">源文件</a>与<a href="./desc/addenda/LDC2009T02_trans.png" rel="nofollow">译文文件</a>图像。 部分内容 © 2005 中国中央电视台，© 2005 凤凰卫视，© 2005 - 2007, 2009 宾夕法尼亚大学理事会。

提供机构：

Linguistic Data Consortium

创建时间：

2020-11-30

搜集汇总

数据集介绍