遇见数据集

yanaiela/tne

收藏
Hugging Face2024-01-18 更新2024-06-15 收录
官方服务:

资源简介:

--- annotations_creators: - crowdsourced language_creators: - found language: - en license: - mit multilinguality: - monolingual size_categories: - 1K<n<10K source_datasets: - original task_categories: - text-retrieval task_ids: - document-retrieval pretty_name: Text-based NP Enrichment dataset_info: features: - name: id dtype: string - name: text dtype: string - name: tokens sequence: string - name: nps list: - name: text dtype: string - name: first_char dtype: int32 - name: last_char dtype: int32 - name: first_token dtype: int32 - name: last_token dtype: int32 - name: id dtype: string - name: np_relations list: - name: anchor dtype: string - name: complement dtype: string - name: preposition dtype: class_label: names: '0': about '1': for '2': with '3': from '4': among '5': by '6': 'on' '7': at '8': during '9': of '10': member(s) of '11': in '12': after '13': under '14': to '15': into '16': before '17': near '18': outside '19': around '20': between '21': against '22': over '23': inside - name: complement_coref_cluster_id dtype: string - name: coref list: - name: id dtype: string - name: members sequence: string - name: np_type dtype: class_label: names: '0': standard '1': time/date/measurement '2': idiomatic - name: metadata struct: - name: annotators struct: - name: coref_worker dtype: int32 - name: consolidator_worker dtype: int32 - name: np-relations_worker sequence: int32 - name: url dtype: string - name: source dtype: string splits: - name: train num_bytes: 41308170 num_examples: 3988 - name: validation num_bytes: 5495419 num_examples: 500 - name: test num_bytes: 2203716 num_examples: 500 - name: test_ood num_bytes: 2249352 num_examples: 509 download_size: 14194578 dataset_size: 51256657 --- # Dataset Card for Text-based NP Enrichment ## Table of Contents - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Supported Tasks](#supported-tasks-and-leaderboards) - [Languages](#languages) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-instances) - [Data Splits](#data-instances) - [Dataset Creation](#dataset-creation) - [Curation Rationale](#curation-rationale) - [Source Data](#source-data) - [Annotations](#annotations) - [Personal and Sensitive Information](#personal-and-sensitive-information) - [Considerations for Using the Data](#considerations-for-using-the-data) - [Social Impact of Dataset](#social-impact-of-dataset) - [Discussion of Biases](#discussion-of-biases) - [Other Known Limitations](#other-known-limitations) - [Additional Information](#additional-information) - [Dataset Curators](#dataset-curators) - [Licensing Information](#licensing-information) - [Citation Information](#citation-information) ## Dataset Description - **Homepage:** https://yanaiela.github.io/TNE/ - **Repository:** https://github.com/yanaiela/TNE - **Paper:** https://arxiv.org/abs/2109.12085 - **Leaderboard:** [TNE OOD](https://leaderboard.allenai.org/tne-ood/submissions/public) [TNE](https://leaderboard.allenai.org/tne/submissions/public) - **Point of Contact:** [Yanai Elazar](mailto:yanaiela@gmail.com) ### Dataset Summary Text-based NP Enrichment (TNE) is a natural language understanding (NLU) task, which focus on relations between noun phrases (NPs) that can be mediated via prepositions. The dataset contains 5,497 documents, annotated exhaustively with all possible links between the NPs in each document. The main data comes from WikiNews, which is used for train/dev/test. We also collected an additional set of 509 documents to serve as out of distribution (OOD) data points, from the Book Corpus, IMDB reviews and Reddit. ### Supported Tasks and Leaderboards The data contain both the main data for the TNE task, as well as coreference resolution data. There are two leaderboards for the TNE data, one for the standard test set, and another one for the OOD test set: - [TNE Leaderboard](https://leaderboard.allenai.org/tne/submissions/public) - [TNE OOD Leaderboard](https://leaderboard.allenai.org/tne-ood/submissions/public) ### Languages The text in the dataset is in English, as spoken in the different domains we include. The associated BCP-47 code is `en`. ## Dataset Structure ### Data Instances The original files are in a jsonl format, containing a dictionary of a single document, in each line. Each document contain a different amount of labels, due to the different amount of NPs. The test and ood splits come without the annotated labels. ### Data Fields A document consists of: * `id`: a unique identifier of a document, beginning with `r` and followed by a number * `text`: the text of the document. The title and subtitles (if exists) are separated with two new lines. The paragraphs are separated by a single new line. * `tokens`: a list of string, containing the tokenized tokens * `nps`: a list of dictionaries, containing the following entries: * `text`: the text of the np * `start_index`: an integer indicating the starting index in the text * `end_index`: an integer indicating the ending index in the text * `start_token`: an integer indicating the first token of the np out of the tokenized tokens * `end_token`: an integer indicating the last token of the np out of the tokenized tokens * `id`: the id of the np * `np_relations`: these are the relation labels of the document. It is a list of dictionaries, where each dictionary contains: * `anchor`: the id of the anchor np * `complement`: the id of the complement np * `preposition`: the preposition that links between the anchor and the complement. This can take one out of 24 pre-defined preposition (23 + member(s)-of) * `complement_coref_cluster_id`: the coreference id, which the complement is part of. * `coref`: the coreference labels. It contains a list of dictionaries, where each dictionary contains: * `id`: the id of the coreference cluster * `members`: the ids of the nps members of such cluster * `np_type`: the type of cluster. It can be either * `standard`: regular coreference cluster * `time/date/measurement`: a time / date / measurement np. These will be singletons. * `idiomatic`: an idiomatic expression * `metadata`: metadata of the document. It contains the following: * `annotators`: a dictionary with anonymized annotators id * `coref_worker`: the coreference worker id * `consolidator_worker`: the consolidator worker id * `np-relations_worker`: the np relations worker id * `url`: the url where the document was taken from (not always existing) * `source`: the original file name where the document was taken from ### Data Splits The dataset is spread across four files, for the four different splits: train, dev, test and test_ood. Additional details on the data statistics can be found in the [paper](https://arxiv.org/abs/2109.12085) ## Dataset Creation ### Curation Rationale TNE was build as a new task for language understanding, focusing on extracting relations between nouns, moderated by prepositions. ### Source Data #### Initial Data Collection and Normalization [Needs More Information] #### Who are the source language producers? [Needs More Information] ### Annotations #### Annotation process [Needs More Information] #### Who are the annotators? [Needs More Information] ### Personal and Sensitive Information [Needs More Information] ## Considerations for Using the Data ### Social Impact of Dataset [Needs More Information] ### Discussion of Biases [Needs More Information] ### Other Known Limitations [Needs More Information] ## Additional Information ### Dataset Curators The dataset was created by Yanai Elazar, Victoria Basmov, Yoav Goldberg, Reut Tsarfaty, during work done at Bar-Ilan University, and AI2. ### Licensing Information The data is released under the MIT license. ### Citation Information ```bibtex @article{tne, author = {Elazar, Yanai and Basmov, Victoria and Goldberg, Yoav and Tsarfaty, Reut}, title = "{Text-based NP Enrichment}", journal = {Transactions of the Association for Computational Linguistics}, year = {2022}, } ``` ### Contributions Thanks to [@yanaiela](https://github.com/yanaiela), who is also the first author of the paper, for adding this dataset.

提供机构:
yanaiela
原始信息汇总

数据集概述

数据集基本信息

  • 数据集名称: Text-based NP Enrichment
  • 语言: 英语
  • 许可证: MIT
  • 数据集大小: 5,497 文档
  • 数据集类型: 单语种
  • 数据集来源: 原始数据
  • 任务类别: 文本检索
  • 任务ID: 文档检索

数据集结构

特征

  • id: 字符串类型,文档的唯一标识符
  • text: 字符串类型,文档的文本内容
  • tokens: 字符串序列,分词后的标记
  • nps: 列表类型,包含以下字段:
    • text: 字符串类型,名词短语的文本
    • first_char: 整数类型,名词短语在文本中的起始字符位置
    • last_char: 整数类型,名词短语在文本中的结束字符位置
    • first_token: 整数类型,名词短语在分词后的起始标记位置
    • last_token: 整数类型,名词短语在分词后的结束标记位置
    • id: 字符串类型,名词短语的唯一标识符
  • np_relations: 列表类型,包含以下字段:
    • anchor: 字符串类型,锚点名词短语的标识符
    • complement: 字符串类型,补语名词短语的标识符
    • preposition: 分类标签类型,介词,包含24种预定义的介词
    • complement_coref_cluster_id: 字符串类型,补语名词短语的共指聚类标识符
  • coref: 列表类型,包含以下字段:
    • id: 字符串类型,共指聚类的标识符
    • members: 字符串序列,共指聚类的成员名词短语标识符
    • np_type: 分类标签类型,共指聚类的类型,包含三种:标准、时间/日期/测量、习语
  • metadata: 结构体类型,包含以下字段:
    • annotators: 结构体类型,包含以下字段:
      • coref_worker: 整数类型,共指标注者ID
      • consolidator_worker: 整数类型,合并标注者ID
      • np-relations_worker: 整数序列,名词短语关系标注者ID
    • url: 字符串类型,文档来源的URL
    • source: 字符串类型,文档来源的原始文件名

数据分割

  • train: 3988 个样本,41308170 字节
  • validation: 500 个样本,5495419 字节
  • test: 500 个样本,2203716 字节
  • test_ood: 509 个样本,2249352 字节

数据集创建

  • 注释创建者: 众包
  • 语言创建者: 发现
  • 数据集大小类别: 1K<n<10K
  • 源数据集: 原始数据

数据集下载

  • 下载大小: 14194578 字节
  • 数据集大小: 51256657 字节
搜集汇总
数据集介绍
yanaiela/tne 数据集图片
构建方式
Text-based NP Enrichment(TNE)数据集旨在推动自然语言理解中名词短语间介词介导关系的建模研究。该数据集主要源自WikiNews语料,通过众包方式对文档中所有名词短语之间的潜在关联进行穷尽式标注,涵盖训练、验证与测试三部分。此外,为增强模型的泛化能力,研究团队还从Book Corpus、IMDB评论及Reddit社区中额外收集了509篇文档,构建出分布外测试集。标注过程涉及核心指代消解与名词短语关系标注,其中介词关系涵盖24种预定义类别。数据集经过严格的众包审核与整合流程,确保标注质量与一致性。
特点
TNE数据集具备鲜明的结构化特征与细粒度标注优势。每篇文档包含完整文本、词元序列、名词短语位置信息及介词关系三元组(锚点、补语、介词类型),并辅以核心指代簇及其类型标注(标准、时间/日期/测量、习语)。数据规模适中,训练集含3988例,验证集与测试集各500例,分布外测试集509例,适合作为中等规模基准任务。其独特的分布外设置使得模型在跨域场景下的鲁棒性评估成为可能,弥补了现有数据集在领域迁移评测上的不足。
使用方法
该数据集以JSONL格式存储,每行对应一篇文档的完整字典结构,便于直接加载与解析。使用时,用户可依据预定义的字段名称提取文本、词元、名词短语及其关系标签。测试集与分布外测试集不提供标注,适用于模型推理评估。数据集支持文本检索任务,并已在Allen AI平台上设有公开排行榜,便于研究者比较模型在标准测试集与分布外测试集上的表现。推荐结合HuggingFace Datasets库进行快速加载,或参照官方GitHub仓库中的示例代码进行定制化开发。
背景与挑战
背景概述
文本型名词短语丰富化(Text-based NP Enrichment, TNE)数据集由Yanai Elazar、Victoria Basmov、Yoav Goldberg及Reut Tsarfaty于2021年在巴伊兰大学与艾伦人工智能研究所(AI2)联合创建,旨在推动自然语言理解中名词短语间介词介导关系的深层语义解析。该数据集聚焦于细粒度的名词短语关系抽取任务,突破传统语义角色标注的局限,通过枚举文档内所有可能的NP间链接,为模型提供了更全面的语义关联视角。TNE包含5,497篇来自维基新闻的标注文档,并额外引入509篇来自书籍语料库、IMDB评论及Reddit的跨域样本以评估泛化能力。其发布不仅填补了介词关系结构化标注的空白,更催生了标准化与域外测试两大排行榜,成为评估语言模型名词短语理解能力的重要基准。
当前挑战
TNE数据集面临的挑战多维且深刻。在领域问题层面,其核心任务要求模型精准识别介词(如“about”、“among”等24类)所隐含的语义关系,这需要超越表面句法匹配的深层推理能力,尤其当名词短语存在指代消解或习语性表达时,关系判定复杂度剧增。此外,跨域泛化构成显著瓶颈——模型在维基新闻等正式语料上训练后,面对IMDB评论的口语化表达或Reddit的碎片化文本时,性能往往骤降,暴露出对非标准语言结构的脆弱性。在构建过程中,数据标注面临主观歧义挑战:不同标注者对介词关系边界的理解可能存在差异,例如“member(s) of”类复合介词的范畴界定需严格规范;同时,核心指代簇的层级性标注(如时间/日期/度量短语的独立处理)要求标注者具备高度一致的判断标准,这增加了人工校验与一致性维护的难度。
常用场景
经典使用场景
在自然语言理解领域,名词短语间的语义关系解析一直是核心难题,尤其是那些由介词介导的隐性关联。Text-based NP Enrichment(TNE)数据集正是为此而生,它提供了5,497篇来自维基新闻等来源的文档,其中详尽标注了所有名词短语之间的介词关系,涵盖23种常见介词及特殊类别。该数据集最经典的使用场景是作为文本级名词短语关系抽取任务的基准,研究人员可据此训练模型,从非结构化文本中精准识别诸如“the book on the table”中由“on”所连接的实体关系,从而推动细粒度语义理解技术的发展。
衍生相关工作
TNE数据集的发布催生了一系列具有影响力的相关工作。在模型架构层面,研究者基于其标注体系提出了专门针对介词关系编码的图神经网络模型,将名词短语视为节点,介词视为边,从而捕捉文档级的结构化语义。此外,该数据集被用于评估预训练语言模型(如BERT、RoBERTa)在细粒度关系理解上的局限性,揭示了当前模型在分布外场景下泛化能力不足的痛点,进而激发了针对OOD鲁棒性的改进方法。同时,TNE还促进了跨任务联合学习的研究,将核心指代消解与名词短语关系抽取相结合,为统一语义解析框架提供了新的基准方向。
数据集最近研究
最新研究方向
在自然语言理解领域,名词短语间介词中介关系的精细化建模正成为前沿焦点。yanaiela/tne 数据集聚焦于文本中名词短语通过介词(如of、in、with等24种类型)建立的语义关联,这一研究方向突破了传统关系抽取的范畴,深入挖掘了介词所承载的微妙语义纽带。该数据集基于维基新闻构建,并引入来自书籍语料库、IMDB评论及Reddit的域外样本,为评估模型在分布偏移下的泛化能力提供了严苛基准。其与共指消解任务的融合标注,更推动了多任务联合学习的发展,对提升机器阅读理解、信息检索等应用的细粒度语义解析能力具有里程碑意义。
以上内容由遇见数据集搜集并总结生成
二维码
社区交流群
二维码
科研交流群
商业服务