Penn Discourse Treebank Version 3.0

<h3>Introduction</h3><br>
<p>Penn Discourse Treebank (PDTB) Version 3.0 is the third release in the Penn Discourse Treebank project, the goal of which is to annotate the Wall Street Journal (WSJ) section of Treebank-2 (<a href="http://catalog.ldc.upenn.edu/LDC95T7" rel="nofollow">LDC95T7</a>) with discourse relations. Penn Discourse Treebank Version 2 (<a href="../../../LDC2008T05" rel="nofollow">LDC2008T05</a>) contains over 40,600 tokens of annotated relations. In Version 3, an additional 13,000 tokens were annotated, certain pairwise annotations were standardized, new senses were included and the corpus was subject to a series of consistency checks. Details concerning the development of PDTB Version 3.0 can be found in the documentation accompanying this release.</p><br>
<p>Largely because the PDTB project was based on the idea that discourse relations are grounded in an identifiable set of explicit words or phrases (discourse connectives) or simply in the adjacency of two sentences, the PTDB has been used by many researchers in the natural language processing community and more recently, by researchers in psycholinguistics. It has also stimulated the development of similar resources in other languages and domains.</p><br>
<h3>Data</h3><br>
<p>Annotations are provided in the form of separate text files (<em>standoff annotation</em>) that are byte-indexed into the raw WSJ text files in Treebank-2. The raw WSJ files are also included in this release. All text files are plain text, encoded in UTF-8.</p><br>
<p>This corpus contains two tools: (1) The Annotator, used for annotation and adjudication, and which can also be used for viewing the corpus; and (2) The Conversion Tool for converting Version 2 annotation files into the Version 3 format.</p><br>
<p>The documentation directory contains a manual describing what is new in Version 3 and how Version 3 differs from Version 2; the methods and guidelines used in annotating PDTB Version 3; and a range of statistics on the tokens, including the frequency of each connective, its sense labels and its modifiers. More information about the corpus and research carried out by the developers and others using the corpus can be found on the <a href="https://www.seas.upenn.edu/~pdtb/">PDTB website</a>.</p><br>
<h3>Samples</h3><br>
<p>One can see samples of the annotation of different types of discourse relations, along with their visualization in the Annotator tool at:</p><br>
<ul><br>
<li><a href="desc/addenda/LDC2019T05_examples.html">Explicit relations</a></li><br>
<li><a href="desc/addenda/LDC2019T05_examples.html#implicit_examples">Implicit relations</a></li><br>
<li><a href="desc/addenda/LDC2019T05_examples.html#altlex_examples">Altlex and AltLexC relations</a></li><br>
<li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">Entity relations</a></li><br>
<li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">Hypophora relations</a></li><br>
<li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">NoRel</a> (annotated only between adjacent sentences within a paragraph that are not linked to each other by a discourse relation)</li><br>
</ul><br>
<h3>Updates</h3><br>
<p>Experiments carried out in Fall 2019 on the intra-sentential discourse relations in the PDTB-3 revealed two problems with the corpus: (1) the final versions of two gold files of "to clause" annotation had not been loaded, and (2) several tokens were inadvertently omitted on the assumption that they were duplicates, when they were not.</p><br>
<p>Repairing these errors, and correcting a mis-labelled token in file wsj_1026, has added another 45 implicit intra-sentential relations to the corpus. Counts in the Annotation Manual have been adjusted to take these additional tokens into account. &nbsp;Specific changes/additions are recorded in the file "pdtb3-revision-jan-2020.txt". Downloads after February 3, 2020 contain the updated corpus.</p><br>
<h3>Acknowledgment</h3><br>
<p>This work has been funded by the National Science Foundation, under grant NSF IIS 1422186 to the University of Pennsylvania and grant NSF IIS 1421067 to the University of Wisconsin, Milwaukee. The content of this publication does not necessarily reflect the position or policy of the Government, and no official endorsement should be inferred.</p></br>
Portions © 1987-1989 Dow Jones & Company, Inc., © 2008, 2012, 2019 The Penn Discourse Treebank Group, © 2008, 2012, 2019 Trustees of the University of Pennsylvania

<h3>引言</h3><br><p>佩恩话语树库（Penn Discourse Treebank，PDTB）3.0版是佩恩话语树库项目的第三版发布成果，其项目目标是为树库2（Treebank-2）中的《华尔街日报》（Wall Street Journal，WSJ）语料部分添加话语关系标注，对应编号为<a href="http://catalog.ldc.upenn.edu/LDC95T7" rel="nofollow">LDC95T7</a>。佩恩话语树库2.0版（编号<a href="../../../LDC2008T05" rel="nofollow">LDC2008T05</a>）已标注超过40600个话语关系Token。在3.0版中，新增标注了13000个话语关系Token，统一了部分成对标注规范，新增了话语关系语义类别，并对语料库开展了一系列一致性校验工作。有关PDTB 3.0版的开发细节，可查阅本发布包附带的官方文档。</p><br><p>由于PDTB项目的核心设计理念为：话语关系可依托一组可识别的显性词汇/短语（即话语连接词），或是仅依托两个句子的邻接关系进行标注，因此该语料库已被自然语言处理领域的众多研究者采用，近期更被心理语言学领域的研究者所使用。同时，该项目也推动了其他语言与领域内同类话语标注语料库的建设。</p><br><h3>数据</h3><br><p>标注数据以独立文本文件（外置标注文件，standoff annotation）的形式提供，通过字节索引关联至树库2中原始WSJ语料的文本文件。本发布包同时包含原始WSJ语料文件，所有文本文件均为UTF-8编码的纯文本格式。</p><br><p>本语料库附带两款工具：其一为标注与审核工具Annotator，同时可用于语料库的可视化浏览；其二为格式转换工具，用于将2.0版的标注文件转换为3.0版格式。</p><br><p>文档目录中包含一份官方手册，内容涵盖3.0版的更新内容、与2.0版的差异细节、PDTB 3.0版的标注方法与规范，以及各类Token统计数据，包括各连接词的出现频次、语义标签及其修饰成分信息。有关本语料库的更多信息，以及开发者与其他使用者开展的相关研究，可访问<a href="https://www.seas.upenn.edu/~pdtb/">PDTB官方网站</a>获取。</p><br><h3>示例</h3><br><p>可通过以下链接查看各类话语关系的标注示例，以及这些示例在Annotator工具中的可视化效果：</p><br><ul><br><li><a href="desc/addenda/LDC2019T05_examples.html">显性话语关系</a></li><br><li><a href="desc/addenda/LDC2019T05_examples.html#implicit_examples">隐性话语关系</a></li><br><li><a href="desc/addenda/LDC2019T05_examples.html#altlex_examples">Altlex与AltLexC关系</a></li><br><li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">实体关系</a></li><br><li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">设问关系（Hypophora）</a></li><br><li><a href="desc/addenda/LDC2019T05_examples.html#entrel_norel_examples">NoRel</a>（仅标注于段落内未通过话语关系关联的相邻句子之间）</li><br></ul><br><h3>更新说明</h3><br><p>2019年秋季针对PDTB-3的句内话语关系开展的实验发现了语料库中的两处问题：其一，两处“to从句”标注的标准金标准文件最终版本未被加载；其二，部分Token因被误判为重复内容而被意外遗漏，实则并非重复。</p><br><p>修复上述错误并更正wsj_1026文件中一处标注错误的Token后，语料库新增了45个句内隐性话语关系。标注手册中的统计数据已针对新增的Token进行了调整。具体的变更与新增内容记录于文件"pdtb3-revision-jan-2020.txt"中。2020年2月3日之后的下载包均包含更新后的语料库。</p><br><h3>致谢</h3><br><p>本项目由美国国家科学基金会资助，宾夕法尼亚大学项目编号为NSF IIS 1422186，威斯康星大学密尔沃基分校项目编号为NSF IIS 1421067。本出版物内容不代表美国政府的立场或政策，也不应被视为获得官方背书。</p><br><p>部分内容 © 1987-1989 道琼斯公司（Dow Jones & Company, Inc.），© 2008、2012、2019 佩恩话语树库工作组，© 2008、2012、2019 宾夕法尼亚大学托管委员会</p>

搜集汇总

背景与挑战

背景概述

Penn Discourse Treebank Version 3.0是基于Treebank-2《华尔街日报》文本的话语关系标注数据集，是该项目第三个版本，新增了13,000个标注并进行了标准化处理。数据以独立文本文件形式提供，包含原始文本和标注工具，广泛应用于自然语言处理和心理学研究，并在2019年更新修复了标注错误。