LLukas22/NLQuAD

Name: LLukas22/NLQuAD
Creator: LLukas22
Published: 2022-12-23 13:04:58
License: 暂无描述

Hugging Face2022-12-23 更新2024-03-04 收录

下载链接：

https://hf-mirror.com/datasets/LLukas22/NLQuAD

下载链接

链接失效反馈

官方服务：

资源简介：

--- pretty_name: NLQuAD language: - en license: - cc-by-3.0 size_categories: - 10K<n<100K multilinguality: - monolingual task_ids: - extractive-qa dataset_info: features: - name: title dtype: string - name: date dtype: string - name: paragraphs list: - name: context dtype: string - name: qas list: - name: answers list: - name: answer_end dtype: int64 - name: answer_start dtype: int64 - name: text dtype: string - name: id dtype: string - name: question dtype: string splits: - name: train num_bytes: 72036724 num_examples: 10259 - name: test num_bytes: 9045482 num_examples: 1280 - name: validation num_bytes: 8876137 num_examples: 1280 download_size: 0 dataset_size: 89958343 --- # Dataset Card for "NLQuAD" ## Table of Contents - [Table of Contents](#table-of-contents) - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-fields) - [Data Splits](#data-splits) - [Additional Information](#additional-information) - [Licensing Information](#licensing-information) - [Citation Information](#citation-information) ## Dataset Description - **Homepage:** [https://github.com/ASoleimaniB/NLQuAD](https://github.com/ASoleimaniB/NLQuAD) - **Paper: https://aclanthology.org/2021.eacl-main.106/** - **Size of the generated dataset:** 89.95 MB ### Dataset Summary This is a copy of the original NLQuAD dataset distributed via [Github](https://github.com/ASoleimaniB/NLQuAD). NLQuAD is a non-factoid long question answering dataset from BBC news articles. NLQuAD’s question types and the long length of its context documents as well as answers, make it a challenging real-world task. NLQuAD consists of news articles as context documents, interrogative sub-headings in the articles as questions, and body paragraphs corresponding to the sub-headings as contiguous answers to the questions. NLQuAD contains 31k non-factoid questions and long answers collected from 13k BBC news articles. See example articles in BBC [1](https://www.bbc.com/news/world-asia-china-51230011), [2](https://www.bbc.com/news/world-55709428). We automatically extract target answers because annotating for non-factoid long QA is extremely challenging and costly. ## Dataset Structure ### Data Instances An example of 'train' looks as follows. ```json { "title": "Khashoggi murder: Body 'dissolved in acid'", "date": "2 November 2018", "paragraphs":[ { "context": "A top Turkish official, presidential adviser Yasin Aktay, has said ....", "qas":[ { "question":"What was said in the crown prince's alleged phone call?", "id":"0_0", "answers":[ { "text":"During the call with President Donald Trump\'s son-in-law Jared Kushner and national ....", "answer_start":1352, "answer_end": 2108, } ] }, { "question":"What has the investigation found so far?", "id":"0_1", "answers":[ { "text":"There is still no consensus on how Khashoggi died. He entered ....", "answer_start":2109, "answer_end": 3128, } ] }, ] } ] } ``` ### Data Fields The data fields are the same among all splits. - `title`: a `string` feature. - `date`: a `string` feature. - `paragraphs`: a list feature containing dictionaries: - `context`: a `string` feature. - `qas`: a list feature containing dictionaries: - `question`: a `string` feature. - `id`: a `string` feature. - `answers`: a list feature containing dictionaries: - `text`: a `string` feature. - `answer_start`: a `int32` feature. - `answer_end`: a `int32` feature ### Data Splits | name |train|test|validation| |----------|----:|----:|---------:| | |10259| 1280| 1280| ## Additional Information ### Licensing Information This dataset is distributed under the [CC BY-NC](https://creativecommons.org/licenses/by-nc/3.0/) licence providing free access for non-commercial and academic usage. ### Citation Information BibTeX: ```json @inproceedings{soleimani-etal-2021-nlquad, title = "{NLQ}u{AD}: A Non-Factoid Long Question Answering Data Set", author = "Soleimani, Amir and Monz, Christof and Worring, Marcel", booktitle = "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume", month = apr, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2021.eacl-main.106", doi = "10.18653/v1/2021.eacl-main.106", pages = "1245--1255", abstract = "We introduce NLQuAD, the first data set with baseline methods for non-factoid long question answering, a task requiring document-level language understanding. In contrast to existing span detection question answering data sets, NLQuAD has non-factoid questions that are not answerable by a short span of text and demanding multiple-sentence descriptive answers and opinions. We show the limitation of the F1 score for evaluation of long answers and introduce Intersection over Union (IoU), which measures position-sensitive overlap between the predicted and the target answer spans. To establish baseline performances, we compare BERT, RoBERTa, and Longformer models. Experimental results and human evaluations show that Longformer outperforms the other architectures, but results are still far behind a human upper bound, leaving substantial room for improvements. NLQuAD{'}s samples exceed the input limitation of most pre-trained Transformer-based models, encouraging future research on long sequence language models.", } ```

提供机构：

LLukas22

原始信息汇总

数据集概述

数据集名称

名称: NLQuAD

语言

语言: 英语 (en)

许可证

许可证: CC-BY-3.0

数据集大小

大小类别: 10K<n<100K
生成数据集大小: 89.95 MB

多语言性

多语言性: 单语种

任务类型

任务ID: extractive-qa

数据集结构

数据实例

示例: 包含标题、日期和段落，每个段落包含上下文和问题答案对。

数据字段

标题 (string): 文章标题。
日期 (string): 文章日期。
段落 (list): 包含以下字段：
- 上下文 (string): 段落内容。
- 问题答案对 (list): 包含以下字段：
  - 问题 (string): 问题文本。
  - ID (string): 问题ID。
  - 答案 (list): 包含以下字段：
    - 文本 (string): 答案文本。
    - 答案开始 (int64): 答案在文本中的开始位置。
    - 答案结束 (int64): 答案在文本中的结束位置。

数据分割

训练集: 10259个实例，大小为72036724字节。
测试集: 1280个实例，大小为9045482字节。
验证集: 1280个实例，大小为8876137字节。

数据集详情

数据集描述: NLQuAD是一个非事实性长问题回答数据集，源自BBC新闻文章，包含31k非事实性问题和长答案，收集自13k BBC新闻文章。
数据集结构: 数据集包含新闻文章作为上下文文档，文章中的疑问副标题作为问题，以及与副标题对应的正文段落作为连续答案。
数据集特点: 由于非事实性长QA的标注极其挑战且成本高，目标答案是自动提取的。

搜集汇总

数据集介绍

构建方式

NLQuAD数据集源自BBC新闻文章，旨在解决非事实型长文本问答这一具有挑战性的现实任务。其构建方式独具匠心：以新闻文章中的疑问性副标题作为问题，而对应的正文段落则被自动提取为连续答案。由于人工标注非事实型长答案极为困难且成本高昂，研究者采用自动化方法从文章中提取目标答案，从而构建了一个包含约3.1万个问题和1.3万篇新闻文章的大规模数据集。

特点

该数据集的核心特点在于其非事实型问题的设计，这些问题无法通过短文本片段回答，而需要多句描述性答案或观点，体现了文档级语言理解的需求。此外，上下文文档和答案的长度远超传统问答数据集，增加了任务难度。为克服传统F1分数在长答案评估中的局限性，NLQuAD引入了交并比（IoU）作为位置敏感的评价指标，更精确地衡量预测与目标答案的重叠程度。

使用方法

数据集划分为训练集（10259例）、测试集（1280例）和验证集（1280例），包含标题、日期、段落上下文、问题及答案起始与结束位置等字段。使用时，可直接加载HuggingFace上的数据集，通过标准化接口访问各拆分。鉴于样本长度超过多数预训练模型的输入限制，推荐采用Longformer等长序列处理架构进行建模，以充分发挥数据集在推动长序列语言模型研究方面的价值。

背景与挑战

背景概述

在自然语言处理领域，抽取式问答系统长期聚焦于短文本片段的事实性查询，然而现实世界中大量问题属于非事实性、需要多句描述的长篇问答。为弥合这一鸿沟，Amir Soleimani、Christof Monz与Marcel Worring于2021年欧洲计算语言学协会会议上提出了NLQuAD数据集。该数据集依托BBC新闻文章，将文章中的疑问式副标题作为问题，对应段落作为连续答案，构建了包含31,000个非事实性长问题与13,000篇新闻文章的语料库。其核心研究问题在于推动文档级语言理解，挑战传统模型对长上下文与复杂语义的建模能力。NLQuAD的发布为长序列语言模型研究提供了基准，其影响力体现在促使学界重新审视评估指标，并推动了如Longformer等长文本架构的发展。

当前挑战

NLQuAD所解决的领域问题在于非事实性长问答的复杂性：问题通常需要多句推理而非简单实体抽取，且答案跨度长、位置敏感，传统F1评分无法准确衡量预测与真实答案的重叠质量。为此，数据集引入交并比（IoU）作为评估指标，但现有模型如BERT、RoBERTa仍受限于输入长度，Longformer虽表现更优，与人类上限仍存在显著差距。构建过程中，挑战尤为突出：人工标注非事实性长答案成本极高且难度大，因此团队采用自动化方法，从新闻文章结构中间接提取问题与答案对，这虽提升了规模，但可能引入噪声，如副标题与段落间的语义不完全对齐，增加了模型训练的难度。

常用场景

经典使用场景

NLQuAD作为首个面向非事实型长文本问答的数据集，其核心应用场景在于评估和推动机器对长篇新闻文档的深度语义理解能力。该数据集以BBC新闻文章为上下文，将文章中的疑问式副标题作为问题，对应的正文段落作为答案，构建了超过3.1万个非事实型问答对。此类问题无法通过简单的实体抽取或短文本匹配来回答，要求模型具备跨句推理、观点归纳和上下文连贯性把握等高级语言理解能力，因此成为检验预训练语言模型在长序列、复杂语义场景下表现的重要基准。

衍生相关工作

NLQuAD的发布催生了一系列重要学术工作。其论文在EACL 2021上发表后，推动了Longformer、BigBird等长序列Transformer模型的改进与评估，这些模型在NLQuAD上的表现成为衡量长文本处理能力的标准之一。后续研究进一步探索了基于检索增强的生成式问答、段落级注意力机制优化等方向，并衍生出如QuALITY、NarrativeQA等面向长文档理解的数据集。该工作还启发了将IoU指标应用于其他需要位置敏感评估的NLP任务，如文本摘要和篇章级关系抽取。

数据集最近研究