lmqg/qg_jaquad

这是一个名为JaQuAD for question generation的数据集，主要用于日语的问题生成任务。数据集来源于JaQuAD，并且是QG-Bench的一部分，旨在为段落级别的问题生成提供一个统一的基准。数据集的测试集未公开，因此从训练集中随机抽取了测试问题。数据集的结构包括问题、段落、答案等字段，并且每个字段都有特定的用途，如训练问题生成模型。数据集的语言为日语，且数据量在10K到100K之间。

This is a dataset titled *JaQuAD for Question Generation*, primarily intended for Japanese question generation tasks. Derived from the original JaQuAD dataset, it is also a component of QG-Bench, aiming to provide a unified benchmark for paragraph-level question generation. The test split of this dataset is not publicly available, so test questions are randomly sampled from the training set. The dataset structure includes fields such as question, passage, and answer, each serving specific purposes such as training question generation models. The dataset is in Japanese, with a scale ranging from 10K to 100K instances.

原始信息汇总

数据集概述

• 数据集名称: JaQuAD for question generation
• 许可证: cc-by-sa-3.0
• 语言: 日语 (ja)
• 多语言性: 单语种
• 大小: 10K<n<100K
• 源数据集: SkelterLabsInc/JaQuAD
• 任务类别:
  • text-generation
• 任务ID:
  • language-modeling
• 标签:
  • question-generation

数据集描述

• 概述: 该数据集是QG-Bench的一部分，用于段落级问题生成任务。它是从JaQuAD数据集中编译而来，专门用于问题生成任务。测试集未公开发布，因此从训练集中随机抽样了测试问题。
• 支持的任务:
  • question-generation: 用于训练问题生成模型，成功标准通常通过BLEU4/METEOR/ROUGE-L/BERTScore/MoverScore等指标衡量。

数据集结构

• 数据字段:
  • question: 字符串类型
  • paragraph: 字符串类型
  • answer: 字符串类型
  • sentence: 字符串类型
  • paragraph_answer: 字符串类型，答案部分用特殊标记<hl>高亮
  • paragraph_sentence: 字符串类型，包含答案的句子部分用特殊标记<hl>高亮
  • sentence_answer: 字符串类型，答案部分用特殊标记<hl>高亮
• 数据分割:
  • train: 27809
  • validation: 3939
  • test: 3939

引用信息

@inproceedings{ushio-etal-2022-generative, title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration", author = "Ushio, Asahi and Alva-Manchego, Fernando and Camacho-Collados, Jose", booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing", month = dec, year = "2022", address = "Abu Dhabi, U.A.E.", publisher = "Association for Computational Linguistics", }