TC-STAR 2005 Evaluation Package - ASR English

TC-STAR is a European integrated project focusing on Speech-to-Speech Translation (SST). To encourage significant breakthrough in all SST technologies, annual open competitive evaluations are organized. Automatic Speech Recognition (ASR), Spoken Language Translation (SLT) and Text-To-Speech (TTS) are evaluated independently and within an end-to-end system.The first TC-STAR evaluation campaign took place in March 2005. Two core technologies were evaluated during the campaign:• Automatic Speech Recognition (ASR),• Spoken Language Translation (SLT).Each evaluation package includes resources, protocols, scoring tools, results of the official campaign, etc., that were used or produced during the first evaluation campaign. The aim of these evaluation packages is to enable external players to evaluate their own system and compare their results with those obtained during the campaign itself.The speech databases made within the TC-STAR project were validated by SPEX, in the Netherlands, to assess their compliance with the TC-STAR format and content specifications.This package includes the material used for the TC-STAR 2005 Automatic Speech Recognition (ASR) first evaluation campaign for the English language. The same packages are available for both Spanish (ELRA-E0003) and Mandarin (ELRA-E0004) for ASR and for SLT in 3 directions, English-to-Spanish (ELRA-E0005), Spanish-to-English (ELRA-E0006), Chinese-to-English (ELRA-E0007).To be able to chain the components, ASR and SLT evaluation tasks were designed to use common sets of raw data and conditions. Two evaluation tasks, common to ASR and SLT, were selected: EPPS (European Parliament Plenary Sessions) task and VOA (Voice of America) task. This package was used within the EPPS task and consists of 2 data sets:- Development data set: consists of audio recordings of Parliament’s sessions from 25 to 28 October 2004, manually transcribed. Approximately 3.5 hours of recordings were selected and transcribed, corresponding to approximately 35,000 running words in English.- Test data set: consists of audio recordings of Parliament’s sessions from 15 to 18 November 2004. As for the development set, the test data set is made of 3.5 hours (35,000 running words).

TC-STAR系一项聚焦于语音到语音翻译（SST）的欧洲综合性项目。为推动SST技术领域取得显著突破，每年均组织开放式竞争性评估活动。自动语音识别（ASR）、口语语言翻译（SLT）以及文本到语音（TTS）技术均独立于端到端系统中进行评估。首次TC-STAR评估活动于2005年3月举行。在该次活动中，评估了两大核心技术：• 自动语音识别（ASR），• 口语语言翻译（SLT）。每个评估包均包含资源、协议、评分工具、官方活动结果等，这些均在首次评估活动中使用或生成。这些评估包的目的是使外部参与者能够评估自己的系统，并将自己的结果与活动中获得的结果进行比较。在荷兰，SPEX对TC-STAR项目中的语音数据库进行了验证，以评估其是否符合TC-STAR格式和内容规范。本包包括用于TC-STAR 2005年自动语音识别（ASR）首次评估活动的英语材料。相同的包也适用于西班牙语（ELRA-E0003）和普通话（ELRA-E0004）的ASR以及三种方向的SLT，即英语至西班牙语（ELRA-E0005）、西班牙语至英语（ELRA-E0006）、中文至英语（ELRA-E0007）。为了能够串联组件，ASR和SLT评估任务被设计为使用共同的原始数据集和条件。选取了两个对ASR和SLT都通用的评估任务：欧洲议会全体会议（EPPS）任务和美国之音（VOA）任务。本包在EPPS任务中使用，并包含两个数据集：- 开发数据集：包含2004年10月25日至28日议会会议的音频录音，并进行手动转录。约选取了3.5小时的录音并进行转录，对应约35,000个英语运行单词。- 测试数据集：包含2004年11月15日至18日议会会议的音频录音。与开发集类似，测试数据集也由3.5小时（35,000个英语运行单词）组成。