English Web Treebank

<h3>Introduction</h3><br>
<p>English Web Treebank was developed by the Linguistic Data Consortium (LDC) with funding through a gift from Google Inc. It consists of over 250,000 words of English weblogs, newsgroups, email, reviews and question-answers manually annotated for syntactic structure and is designed to allow language technology researchers to develop and evaluate the robustness of parsing methods in those web domains.</p><br>
<h3>Data</h3><br>
<p>This release contains 254,830 word-level tokens and 16,624 sentence-level tokens of webtext in 1174 files annotated for sentence- and word-level tokenization, part-of-speech, and syntactic structure. The data is roughly evenly divided across five genres: weblogs, newsgroups, email, reviews, and question-answers. The files were manually annotated following the sentence-level tokenization guidelines for web text and the word-level tokenization guidelines developed for English treebanks in the <a href="http://projects.ldc.upenn.edu/gale/index.html" rel="nofollow">DARPA GALE</a> project. Only text from the subject line and message body of posts, articles, messages and question-answers were collected and annotated.</p><br>
<p>Weblogs are interactive web sites that display content as discrete entries or posts and allow viewers to comment on entries and engage in discussions. They are typically managed by individuals and use informal or colloquial language. The weblog data in this release was collected by LDC and covers the period 2003-2006.</p><br>
<p>Newsgroups are repositories of online discussions pertaining to a topic or interest area. They consist of threads that in turn contain articles with comments and discussion from group users. The newsgroup data in this release was collected by LDC and covers the period 2003-2006.</p><br>
<p>Email are messages sent to discrete individuals or well defined groups via the TCP-IP Simple Mail Transfer Protocol (SMTP). The email messages in this corpus are a subset of emails sent by Enron Corporation employees during the period 1999-2002. Specifically, those messages are contained in the <a href="http://verbs.colorado.edu/enronsent/" rel="nofollow">Enronsent Corpus</a>, a collection of 96,107 email messages from the sent folders of Enron email users which were processed to remove any content not generated by human users.</p><br>
<p>The reviews in this corpus were gleaned from online reviews of businesses and services on various Google web sites written by individuals. This information was provided to LDC by Google in 2011 the dates of individual reviews are not available.</p><br>
<p>Question-answers are posts from Yahoo!s community-driven question-answering web site, <a href="http://answers.yahoo.com/" rel="nofollow">Yahoo! Answers</a>, where individuals submit and answer questions which may be on any topic. This data was collected in 2011; the dates of individual question-answers were not collected.</p><br>
<h3>Samples</h3></br>
Portions © 2012 Google Inc., © 2011 Yahoo! Inc., © 2012 Trustees of the University of Pennsylvania

<h3>引言</h3><br><p>英语网络树库（English Web Treebank）由语言数据联盟（Linguistic Data Consortium, LDC）开发，资助资金来自谷歌公司（Google Inc.）的一笔捐赠。该数据集包含超过25万词的英文文本，涵盖博客、新闻组、电子邮件、评论与问答内容，所有文本均经过人工句法结构标注，旨在帮助语言技术研究者开发并评估各类句法解析方法在上述网络领域中的鲁棒性。</p><br><h3>数据说明</h3><br><p>本次发布的数据集包含1174个文件中的网络文本，共计254830个词级标记（Token）与16624个句级标记（Token），所有文本均经过句级与词级分词、词性标注以及句法结构标注。数据大致均匀分布在五大文本类型中：博客、新闻组、电子邮件、评论与问答。本次标注遵循了针对网络文本的句级分词指南，以及为<a href="http://projects.ldc.upenn.edu/gale/index.html" rel="nofollow">DARPA GALE</a>项目中的英语树库制定的词级分词指南。仅收集并标注了帖子、文章、消息与问答内容的主题行与正文文本。</p><br><p>博客（Weblogs）是指以独立条目或帖子形式展示内容，并允许访客对条目进行评论、参与讨论的交互式网站，通常由个人运营，语言风格较为非正式且口语化。本次发布的博客数据由LDC收集，覆盖2003年至2006年的文本。</p><br><p>新闻组（Newsgroups）是围绕特定主题或兴趣领域展开在线讨论的资料库，其内容以讨论串形式组织，包含来自组内用户的文章、评论与讨论内容。本次发布的新闻组数据由LDC收集，覆盖2003年至2006年的文本。</p><br><p>电子邮件（Email）是指通过TCP/IP简单邮件传输协议（Simple Mail Transfer Protocol, SMTP）发送给特定个人或明确群组的消息。本语料库中的电子邮件均来自安然公司（Enron Corporation）员工在1999年至2002年间发送的邮件子集，具体而言，这些邮件收录于<a href="http://verbs.colorado.edu/enronsent/" rel="nofollow">Enronsent语料库</a>，该语料库包含96107封来自安然邮件用户已发送文件夹的邮件，且已过滤掉非人工生成的内容。</p><br><p>本语料库中的评论数据均来自个人在谷歌各网站上发布的企业与服务在线评论，该数据集于2011年由谷歌提供给LDC，无法获取单条评论的具体发布日期。</p><br><p>问答（Question-answers）内容来自雅虎社区驱动的问答网站<a href="http://answers.yahoo.com/" rel="nofollow">Yahoo! Answers</a>，用户可在此提交或回答任意主题的问题。该数据集于2011年收集，无法获取单条问答的具体日期。</p><br><h3>示例</h3></br>部分内容 © 2012 Google Inc.、© 2011 Yahoo! Inc.、© 2012 宾夕法尼亚大学校董会

搜集汇总

背景与挑战

背景概述

English Web Treebank是一个英语网络文本树库数据集，包含超过25万个词和1.6万个句子，覆盖博客、新闻组、电子邮件、评论和问答五种网络文本类型。该数据集由Linguistic Data Consortium于2012年发布，手动标注了句法和词法结构，旨在支持语言技术研究，特别是解析方法的鲁棒性评估，适用于词性标注、解析和问答系统等应用。