DEFT Corpus

DEFT corpus1由来自两个不同数据源的注释内容组成: 1) 来自公开获得的美国证券交易委员会埃德加 (SEC) 数据库的各种2017 SEC合同文件的2,443句子 (5,324,430令牌)，和2) 来自https://cnx.org/开源教科书 (由各种作者，4.0 CC授权) 的21,303个句子 (409,253个令牌)，包括生物学，历史，物理学，心理学，经济学，社会学和政府的主题。SEC句子的22% 包含定义，教科书句子的28% 包含定义。我们的整个语料库 (包括两个数据集) 比任何现有的定义提取数据集都要大得多，也要复杂得多 (请参见表1)。

The DEFT Corpus 1 consists of annotated content from two distinct data sources: 1) 2,443 sentences (5,324,430 tokens) of various 2017 SEC contract documents publicly sourced from the U.S. Securities and Exchange Commission (SEC) EDGAR database, and 2) 21,303 sentences (409,253 tokens) from open-source textbooks hosted at https://cnx.org (authored by various contributors, licensed under CC 4.0), covering topics including biology, history, physics, psychology, economics, sociology, and government. 22% of the SEC sentences contain definitions, while 28% of the textbook sentences contain definitions. The entire corpus, including both datasets, is significantly larger and more complex than any existing definition extraction dataset (see Table 1).

搜集汇总

背景与挑战

背景概述

DEFT Corpus是一个用于定义提取任务的大规模语料库，包含来自SEC合同文件和开源教科书的约2.4万个句子，其中约四分之一标注了定义内容。该数据集覆盖多学科主题，采用CC BY-NC-SA 4.0授权协议。