lmqg/qa_harvesting_from_wikipedia_pseudo

这是一个合成的问答数据集，通过微调的QG模型在`lmqg/qa_harvesting_from_wikipedia`数据集上生成，该数据集包含1百万个段落和答案对，最初由[Du and Cardie, 2018]收集，用于基于问答的评估（QAE）以评估[Zhang and Bansal, 2019]提出的问题生成模型。训练集是合成数据，验证集是SQuAD的原始验证集，模型应在此验证集上进行评估。数据集包含通过以下QG模型生成的合成问答数据：`lmqg/bart-base-squad`、`lmqg/bart-large-squad`、`lmqg/t5-small-squad`、`lmqg/t5-base-squad`和`lmqg/t5-large-squad`。数据集的字段包括id、title、context、question和answers。数据集的大小为训练集1,092,142条，验证集10,570条。This is a synthetic question answering (QA) dataset generated using a fine-tuned question generation (QG) model on the `lmqg/qa_harvesting_from_wikipedia` dataset. The original `lmqg/qa_harvesting_from_wikipedia` dataset contains 1 million paragraph-answer pairs, which was initially collected by [Du and Cardie, 2018] for question answering-based evaluation (QAE) to assess the question generation model proposed by [Zhang and Bansal, 2019]. The training set of this synthetic dataset consists of synthetic data, while the validation set is the original SQuAD validation split, and models should be evaluated on this validation set. The synthetic QA data in this dataset is generated by the following QG models: `lmqg/bart-base-squad`, `lmqg/bart-large-squad`, `lmqg/t5-small-squad`, `lmqg/t5-base-squad`, and `lmqg/t5-large-squad`. The dataset includes the following fields: id, title, context, question, and answers. The sizes of the dataset splits are 1,092,142 samples for the training set and 10,570 samples for the validation set.

原始信息汇总

数据集概述

基本信息

• 名称: Synthetic QA dataset
• 许可证: cc-by-4.0
• 语言: 英语 (en)
• 多语言性: 单语种
• 大小: 10K<n<100K

数据来源

• 源数据集: 扩展自 lmqg/qa_harvesting_from_wikipedia
• 数据生成: 使用精细调整的问答生成（QG）模型生成

任务类别

• 任务: 问答
• 任务ID: extractive-qa

数据结构

• 数据字段:
  • id: 字符串类型，标识符
  • title: 字符串类型，段落标题
  • context: 字符串类型，段落内容
  • question: 字符串类型，问题
  • answers: JSON格式，答案

数据分割

• 训练集: 1,092,142条记录
• 验证集: 10,570条记录

引用信息

@inproceedings{ushio-etal-2022-generative, title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration", author = "Ushio, Asahi and Alva-Manchego, Fernando and Camacho-Collados, Jose", booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing", month = dec, year = "2022", address = "Abu Dhabi, U.A.E.", publisher = "Association for Computational Linguistics", }