OntoNotes Release 2.0

<h3>Introduction</h3> <p>The OntoNotes project is a collaborative effort between BBN Technologies, the University of Colorado, the University of Pennsylvania, and the University of Southern California's Information Sciences Institute. The goal of the project is to annotate a large corpus comprising various genres of text (news, conversational telephone speech, weblogs, use net, broadcast, talk shows) in three languages (English, Chinese, and Arabic) with structural information (syntax and predicate argument structure) and shallow semantics (word sense linked to an ontology and coreference). OntoNotes Release 2.0 is a continuation of the OntoNotes project and is supported by the Defense Advanced Research Projects Agency, GALE Program Contract No. HR0011-06-C-0022.</p> <p><a href="http://catalog.ldc.upenn.edu/LDC2007T21" rel="nofollow">OntoNotes Release 1.0 (LDC2007T21)</a> contains 400k words of Chinese newswire data (from Xinhua News Agency and Sinorama Magazine) and 300k words of English newswire data (from the Wall Street Journal). OntoNotes Release 2.0 adds the following to the corpus: 274k words of Chinese broadcast news data (from China Broadcating System, China Central TV, China National Radio, China Television System and Voice of America); and 200k words of English broadcast news data (from ABC, CNN, NBC, Public Radio International and Voice of America). </p> <p>Natural language applications like machine translation, question answering, and summarization currently are forced to depend on impoverished text models like bags of words or n-grams, while the decisions that they are making ought to be based on the meanings of those words in context. That lack of semantics causes problems throughout the applications. Misinterpreting the meaning of an ambiguous word results in failing to extract data, incorrect alignments for translation, and ambiguous language models. Incorrect coreference resolution results in missed information (because a connection is not made) or incorrectly conflated information (due to false connections). OntoNotes builds on two time-tested resources, following the Penn Treebank for syntax and the Penn PropBank for predicate-argument structure. Its semantic representation will include word sense disambiguation for nouns and verbs, with each word sense connected to an ontology, and coreference. The current goals call for annotation of over a million words each of English and Chinese, and half a million words of Arabic over five years.</p> <p>The authors wish to make this resource available to the natural language research community so that decoders for these phenomena can be trained to generate the same structure in new documents. Lessons learned over the years have shown that the quality of annotation is crucial if it is going to be used for training machine learning algorithms. Taking this cue, each layer of annotation in OntoNotes will have at least 90% inter-annotator agreement. Pilot studies have shown that predicate structure, word sense, ontology linking, and coreference can all be annotated rapidly and with better than 90% consistency. </p> <h3>Samples</h3> <p>For an example of the data in this corpus, please examine the following samples</p> <ul> <li><a href="./desc/addenda/LDC2008T04_ch.jpg" rel="nofollow">Chinese</a></li> <li><a href="./desc/addenda/LDC2008T04.jpg" rel="nofollow">English</a></li> </ul><h3>Sponsorship</h3> <p>This work is supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or policy of the Government, and no official endorsement should be inferred.</p> <p>The World is a co-production of Public Radio International and the British Broadcasting Corporation and is produced at WGBH Boston.</p></br>
Portions © 2000-2001 American Broadcasting Company, © 2000-2001 Cable News Network, LP, LLLP, © 2000-2001 China Broadcasting System, © 2000-2001 China Central TV, © 2000-2001 China National Radio, © 2000-2001 China Television System, © 1989 Dow Jones &amp; Company, Inc., © 2000-2001 National Broadcasting, Company, Inc., © 2000-2001 Public Radio International, © 1996-2001 Sinorama Magazine, © 1994-1998 Xinhua News Agency, © 1995, 2005, 2006, 2007, 2008 Trustees of the University of Pennsylvania <br><br>The World is a co-production of Public Radio International and the British Broadcasting Corporation and is produced at WGBH Boston.

<h3>引言</h3> <p>OntoNotes项目是由BBN科技公司（BBN Technologies）、科罗拉多大学（University of Colorado）、宾夕法尼亚大学（University of Pennsylvania）以及南加州大学信息科学研究所（University of Southern California's Information Sciences Institute）联合发起的合作项目。本项目的目标是为涵盖多种文本体裁（新闻、会话电话语音、博客、新闻组（usenet）、广播节目、脱口秀）的大型语料库进行标注，该语料库包含英语、中文、阿拉伯语三种语言，标注内容包括结构化信息（句法与谓词论元结构）以及浅层语义（与本体关联的词义与共指信息）。OntoNotes 2.0版是该项目的延续，其资助方为美国国防高级研究计划局（Defense Advanced Research Projects Agency）GALE项目，合同编号HR0011-06-C-0022。</p> <p><a href="http://catalog.ldc.upenn.edu/LDC2007T21" rel="nofollow">OntoNotes 1.0版（LDC2007T21）</a>包含40万字中文新闻语料（源自新华社（Xinhua News Agency）与Sinorama杂志）以及30万字英文新闻语料（源自《华尔街日报》（Wall Street Journal））。OntoNotes 2.0版在此前语料库基础上新增了以下内容：27.4万字中文广播新闻语料（源自中国广播系统、中国中央电视台（CCTV）、中国国家广播电台、中国电视系统以及美国之音（VOA））；以及20万字英文广播新闻语料（源自美国广播公司（ABC）、美国有线电视新闻网（CNN）、美国全国广播公司（NBC）、国际公共广播电台与美国之音（VOA））。</p> <p>当前，机器翻译、问答系统与文本摘要等自然语言应用往往只能依赖词袋模型或n元语法模型这类较为粗糙的文本模型，而此类应用本应基于词汇在具体语境中的语义做出决策。语义的缺失会给各类应用带来诸多问题：对歧义词汇的语义误判会导致数据提取失败、翻译对齐错误以及语言模型歧义；而共指消解错误则会造成信息遗漏（因未建立正确关联）或信息误合并（因建立了错误关联）。OntoNotes依托两项经过实践验证的资源构建：句法标注沿用宾州树库（Penn Treebank）标准，谓词论元结构标注则参考宾州谓词论元库（Penn PropBank）。其语义表征将涵盖名词与动词的词义消歧任务，每个词义均与本体关联，并包含共指标注。项目当前的目标是在五年内，完成英语、中文语料各超100万字，以及阿拉伯语语料50万字的标注工作。</p> <p>项目团队希望将该资源开放给自然语言研究社区，以便相关研究者可以训练针对这些语言现象的解码器，使其能够在新文档中生成一致的标注结构。多年的研究经验表明，若标注数据用于训练机器学习算法，标注质量至关重要。基于这一原则，OntoNotes的每一层标注都将实现至少90%的标注者间一致性。先导研究证实，谓词结构、词义、本体关联以及共指信息的标注均可快速完成，且一致性优于90%。</p> <h3>示例样本</h3> <p>若需查看本语料库的数据示例，请参阅以下样本</p> <ul> <li><a href="./desc/addenda/LDC2008T04_ch.jpg" rel="nofollow">中文样本</a></li> <li><a href="./desc/addenda/LDC2008T04.jpg" rel="nofollow">英文样本</a></li> </ul><h3>资助说明</h3> <p>本项目部分受美国国防高级研究计划局GALE项目资助，资助编号HR0011-06-1-0003。本文档内容不代表美国政府的立场或政策，也不应被视为获得官方背书。</p> <p>《The World》栏目由国际公共广播电台与英国广播公司联合制作，制作方为波士顿WGBH工作室。</p></br> 部分内容 © 2000-2001 美国广播公司、© 2000-2001 美国有线电视新闻网有限责任合伙公司、© 2000-2001 中国广播系统、© 2000-2001 中国中央电视台、© 2000-2001 中国国家广播电台、© 2000-2001 中国电视系统、© 1989 道琼斯公司、© 2000-2001 美国全国广播公司、© 2000-2001 国际公共广播电台、© 1996-2001 Sinorama杂志、© 1994-1998 新华社、© 1995、2005、2006、2007、2008 宾夕法尼亚大学理事会<br><br> <p>《The World》栏目由国际公共广播电台与英国广播公司联合制作，制作方为波士顿WGBH工作室。</p>

搜集汇总

背景与挑战

背景概述

OntoNotes Release 2.0是一个多语言语料库，包含中文和英文的广播新闻数据，总字数约474k，旨在通过标注句法和浅层语义（如词义消歧和共指消解）支持自然语言处理任务，如信息提取和检索。它是OntoNotes项目的一部分，延续了前序版本，并强调高标注质量（至少90%的标注者一致性）。