TC-STAR 2006 Evaluation Package - SLT Chinese-to-English

TC-STAR is a European integrated project focusing on Speech-to-Speech Translation (SST). To encourage significant breakthrough in all SST technologies, annual open competitive evaluations are organized. Automatic Speech Recognition (ASR), Spoken Language Translation (SLT) and Text-To-Speech (TTS) are evaluated independently and within an end-to-end system.The second TC-STAR evaluation campaign took place in March 2006. Three core technologies were evaluated during the campaign:• Automatic Speech Recognition (ASR),• Spoken Language Translation (SLT),• Text to Speech (TTS).Each evaluation package includes resources, protocols, scoring tools, results of the official campaign, etc., that were used or produced during the second evaluation campaign. The aim of these evaluation packages is to enable external players to evaluate their own system and compare their results with those obtained during the campaign itself.The speech databases made within the TC-STAR project were validated by SPEX, in the Netherlands, to assess their compliance with the TC-STAR format and content specifications.This package includes the material used for the TC-STAR 2006 Spoken Language Translation (SLT) second evaluation campaign for Chinese-to-English translation. The same packages are available for English (ELRA-E0011), Spanish (ELRA-E0012) and Mandarin Chinese (ELRA-E0013) for ASR and for SLT in 2 other directions, English-to-Spanish (ELRA-E0014) and Spanish-to-English (ELRA-E0015).To be able to chain the components, ASR, SLT and TTS evaluation tasks were designed to use common sets of raw data and conditions. Three evaluation tasks, common to ASR, SLT and TTS, were selected: EPPS (European Parliament Plenary Sessions) task, CORTES (Spanish Parliament Sessions) task and VOA (Voice of America) task. The CORTES data were used in addition to the EPPS data to evaluate ASR in Spanish and SLT from Spanish into English.This package was used within the VOA task and consists of 2 data sets:-Development data set: built upon the ASR development data set, in order to enable end-to-end evaluation. Subsets of 25,000 words were selected from the VOA verbatim transcriptions. The source texts were then translated into English by two independent translation agencies. All source text sets and reference translations were formatted using the same SGML DTD that has been used for the NIST Machine Translation evaluations.-Test data set: as for the development set, the same procedure was followed to produce the test data, i.e.: subsets of 25,000 words were selected from the test data set from the manual transcriptions. The source data were then translated into English by two independent agencies

TC-STAR是一项旨在聚焦语音到语音翻译（SST）的欧洲综合项目。为促进SST技术领域的重大突破，项目每年组织举办公开的竞技性评估活动。自动语音识别（ASR）、口语语言翻译（SLT）和文本到语音（TTS）均独立于端到端系统中进行评估。2006年3月，第二次TC-STAR评估活动举行。在此次活动中，对三大核心技术进行了评估：• 自动语音识别（ASR），• 口语语言翻译（SLT），• 文本转语音（TTS）。每个评估包包括资源、协议、评分工具、官方活动结果等，这些均在第二次评估活动中使用或生成。这些评估包的目的是使外部参与者能够评估自己的系统，并将自己的结果与活动中获得的结果进行比较。TC-STAR项目构建的语音数据库由荷兰的SPEX进行验证，以评估其是否符合TC-STAR的格式和内容规范。本包包括用于TC-STAR 2006口语语言翻译（SLT）第二次评估活动的材料，针对中文到英文的翻译。同样的包也适用于英语（ELRA-E0011）、西班牙语（ELRA-E0012）和普通话（ELRA-E0013）的ASR以及两种SLT方向，即英语到西班牙语（ELRA-E0014）和西班牙语到英语（ELRA-E0015）。为了实现组件的链式连接，ASR、SLT和TTS的评估任务被设计为使用相同的原始数据集和条件。选定了三个对ASR、SLT和TTS都通用的评估任务：欧洲议会全体会议（EPPS）任务、西班牙议会会议（CORTES）任务和美国之音（VOA）任务。在EPPS数据的基础上，还使用了CORTES数据来评估西班牙语的ASR和从西班牙语到英语的SLT。本包用于VOA任务，并包括两个数据集：- 开发数据集：基于ASR开发数据集构建，以便进行端到端评估。从VOA的逐字转录中选择了25,000词的子集。源文本随后由两家独立的翻译机构翻译成英文。所有源文本集和参考翻译均使用与NIST机器翻译评估相同的SGML DTD进行格式化。- 测试数据集：与开发集类似，遵循相同的程序生成测试数据，即：从手动转录的测试数据集中选择了25,000词的子集。源数据随后由两家独立的机构翻译成英文。