five

msalnikov/mintaka

收藏
Hugging Face2024-04-18 更新2024-06-11 收录
下载链接：
https://hf-mirror.com/datasets/msalnikov/mintaka
下载链接
链接失效反馈
官方服务：
资源简介：
--- annotations_creators: - expert-generated language_creators: - found language: - en license: - cc-by-4.0 multilinguality: - ar - de - ja - hi - pt - en - es - it - fr size_categories: - 100K<n<1M source_datasets: - https://huggingface.co/datasets/AmazonScience/mintaka task_categories: - question-answering task_ids: - open-domain-qa paperswithcode_id: mintaka pretty_name: Mintaka language_bcp47: - ar-SA - de-DE - ja-JP - hi-HI - pt-PT - en-EN - es-ES - it-IT - fr-FR dataset_info: - config_name: T5-Large-SSM-answers features: - name: id dtype: string - name: lang dtype: string - name: question dtype: string - name: answerText dtype: string - name: category dtype: string - name: complexityType dtype: string - name: questionEntity list: - name: entityType dtype: string - name: label dtype: string - name: mention dtype: string - name: name dtype: string - name: span sequence: int32 - name: answerEntity list: - name: label dtype: string - name: name dtype: string - name: generated_answers sequence: string splits: - name: train num_bytes: 17007018 num_examples: 14000 - name: validation num_bytes: 3009249 num_examples: 2000 - name: test num_bytes: 4805840 num_examples: 4000 download_size: 9102570 dataset_size: 24822107 - config_name: T5-Large-SSM-answers-linked features: - name: id dtype: string - name: lang dtype: string - name: question dtype: string - name: answerText dtype: string - name: category dtype: string - name: complexityType dtype: string - name: questionEntity list: - name: entityType dtype: string - name: label dtype: string - name: mention dtype: string - name: name dtype: string - name: span sequence: int32 - name: answerEntity list: - name: label dtype: string - name: name dtype: string - name: generated_answers sequence: string - name: linked_generated_answers sequence: sequence: string splits: - name: test num_bytes: 7736223 num_examples: 4000 download_size: 2940268 dataset_size: 7736223 - config_name: T5-XL-SSM-answers features: - name: id dtype: string - name: lang dtype: string - name: question dtype: string - name: answerText dtype: string - name: category dtype: string - name: complexityType dtype: string - name: questionEntity list: - name: entityType dtype: string - name: label dtype: string - name: mention dtype: string - name: name dtype: string - name: span sequence: int32 - name: answerEntity list: - name: label dtype: string - name: name dtype: string - name: generated_answers sequence: string splits: - name: train num_bytes: 35216238 num_examples: 28000 - name: validation num_bytes: 2593667 num_examples: 2000 - name: test num_bytes: 5095967 num_examples: 4000 download_size: 14752335 dataset_size: 42905872 - config_name: default features: - name: id dtype: string - name: lang dtype: string - name: question dtype: string - name: answerText dtype: string - name: category dtype: string - name: complexityType dtype: string - name: questionEntity list: - name: name dtype: string - name: entityType dtype: string - name: label dtype: string - name: mention dtype: string - name: span list: int32 - name: answerEntity list: - name: name dtype: string - name: label dtype: string - name: relevant_triplets sequence: sequence: string - name: verbalized_relevant_triplets sequence: string - name: relevant_triplets_g2t dtype: string splits: - name: train num_bytes: 19478013 num_examples: 14000 - name: validation num_bytes: 2791664 num_examples: 2000 - name: test num_bytes: 5572329 num_examples: 4000 download_size: 8236687 dataset_size: 27842006 configs: - config_name: T5-Large-SSM-answers data_files: - split: train path: T5-Large-SSM-answers/train-* - split: validation path: T5-Large-SSM-answers/validation-* - split: test path: T5-Large-SSM-answers/test-* - config_name: T5-Large-SSM-answers-linked data_files: - split: test path: T5-Large-SSM-answers-linked/test-* - config_name: T5-XL-SSM-answers data_files: - split: train path: T5-XL-SSM-answers/train-* - split: validation path: T5-XL-SSM-answers/validation-* - split: test path: T5-XL-SSM-answers/test-* - config_name: default data_files: - split: train path: data/train-* - split: validation path: data/validation-* - split: test path: data/test-* --- # Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering Extended version of original Mintaka dataset with extracted relevan triplets for questions entities (Method from [KAPING](https://arxiv.org/abs/2306.04136v1)) Relevant triplets converted to text by [T5 model tuned on WebNLG dataset](https://huggingface.co/s-nlp/g2t-t5-xl-webnlg) In addition, provided generated answers from [T5 Large SSM](https://huggingface.co/msalnikov/kgqa-mintaka-t5-large-ssm) and [T5 XXL SSM](https://huggingface.co/msalnikov/kgqa-mintaka-t5-xl-ssm-nq) models, tuned on Mintaka in corresponding config names.

annotations_creators: 注释创建者 - expert-generated: 专家生成 language_creators: 语言采集方式 - found: 公开语料爬取 language: 语言 - en: 英语 license: 许可证 - cc-by-4.0: CC BY 4.0（知识共享署名4.0国际许可协议） multilinguality: 多语言覆盖范围 - ar: 阿拉伯语 - de: 德语 - ja: 日语 - hi: 印地语 - pt: 葡萄牙语 - en: 英语 - es: 西班牙语 - it: 意大利语 - fr: 法语 size_categories: 样本规模类别 - 100K<n<1M: 10万 < 样本量 < 100万 source_datasets: 源数据集 - https://huggingface.co/datasets/AmazonScience/mintaka task_categories: 任务类别 - question-answering: 问答 task_ids: 任务子类型 - open-domain-qa: 开放域问答 paperswithcode_id: PapersWithCode 标识符 - mintaka pretty_name: 可视化数据集名称 - Mintaka language_bcp47: 语言BCP47标签 - ar-SA: 沙特阿拉伯阿拉伯语 - de-DE: 德国德语 - ja-JP: 日本日语 - hi-HI: 印度印地语 - pt-PT: 葡萄牙葡萄牙语 - en-EN: 通用英语 - es-ES: 西班牙西班牙语 - it-IT: 意大利意大利语 - fr-FR: 法国法语 dataset_info: - config_name: T5-Large-SSM-answers features: - name: id, dtype: string: 字段id，数据类型：字符串 - name: lang, dtype: string: 字段lang，数据类型：字符串 - name: question, dtype: string: 字段question（问题文本），数据类型：字符串 - name: answerText, dtype: string: 字段answerText（标准答案文本），数据类型：字符串 - name: category, dtype: string: 字段category（问题类别），数据类型：字符串 - name: complexityType, dtype: string: 字段complexityType（问题复杂度类型），数据类型：字符串 - name: questionEntity, list: 字段questionEntity（问题实体列表），包含子字段： - name: entityType, dtype: string: 实体类型，字符串类型 - name: label, dtype: string: 实体标签，字符串类型 - name: mention, dtype: string: 实体文本提及，字符串类型 - name: name, dtype: string: 实体名称，字符串类型 - name: span, sequence: int32: 实体在文本中的位置跨度，32位整数序列 - name: answerEntity, list: 字段answerEntity（答案实体列表），包含子字段： - name: label, dtype: string: 答案实体标签，字符串类型 - name: name, dtype: string: 答案实体名称，字符串类型 - name: generated_answers, sequence: string: 字段generated_answers（生成答案列表），字符串序列 splits: - name: train, num_bytes: 17007018, num_examples: 14000: 训练集：数据量17007018字节，样本量14000 - name: validation, num_bytes: 3009249, num_examples: 2000: 验证集：数据量3009249字节，样本量2000 - name: test, num_bytes: 4805840, num_examples: 4000: 测试集：数据量4805840字节，样本量4000 download_size: 9102570: 总下载大小：9102570字节 dataset_size: 24822107: 数据集总大小：24822107字节 - config_name: T5-Large-SSM-answers-linked features: - name: id, dtype: string: 字段id，数据类型：字符串 - name: lang, dtype: string: 字段lang，数据类型：字符串 - name: question, dtype: string: 字段question（问题文本），数据类型：字符串 - name: answerText, dtype: string: 字段answerText（标准答案文本），数据类型：字符串 - name: category, dtype: string: 字段category（问题类别），数据类型：字符串 - name: complexityType, dtype: string: 字段complexityType（问题复杂度类型），数据类型：字符串 - name: questionEntity, list: 字段questionEntity（问题实体列表），包含子字段： - name: entityType, dtype: string: 实体类型，字符串类型 - name: label, dtype: string: 实体标签，字符串类型 - name: mention, dtype: string: 实体文本提及，字符串类型 - name: name, dtype: string: 实体名称，字符串类型 - name: span, sequence: int32: 实体在文本中的位置跨度，32位整数序列 - name: answerEntity, list: 字段answerEntity（答案实体列表），包含子字段： - name: label, dtype: string: 答案实体标签，字符串类型 - name: name, dtype: string: 答案实体名称，字符串类型 - name: generated_answers, sequence: string: 字段generated_answers（生成答案列表），字符串序列 - name: linked_generated_answers, sequence: sequence: string: 字段linked_generated_answers（链接后生成答案列表），嵌套字符串序列 splits: - name: test, num_bytes: 7736223, num_examples: 4000: 测试集：数据量7736223字节，样本量4000 download_size: 2940268: 总下载大小：2940268字节 dataset_size: 7736223: 数据集总大小：7736223字节 - config_name: T5-XL-SSM-answers features: - name: id, dtype: string: 字段id，数据类型：字符串 - name: lang, dtype: string: 字段lang，数据类型：字符串 - name: question, dtype: string: 字段question（问题文本），数据类型：字符串 - name: answerText, dtype: string: 字段answerText（标准答案文本），数据类型：字符串 - name: category, dtype: string: 字段category（问题类别），数据类型：字符串 - name: complexityType, dtype: string: 字段complexityType（问题复杂度类型），数据类型：字符串 - name: questionEntity, list: 字段questionEntity（问题实体列表），包含子字段： - name: entityType, dtype: string: 实体类型，字符串类型 - name: label, dtype: string: 实体标签，字符串类型 - name: mention, dtype: string: 实体文本提及，字符串类型 - name: name, dtype: string: 实体名称，字符串类型 - name: span, sequence: int32: 实体在文本中的位置跨度，32位整数序列 - name: answerEntity, list: 字段answerEntity（答案实体列表），包含子字段： - name: label, dtype: string: 答案实体标签，字符串类型 - name: name, dtype: string: 答案实体名称，字符串类型 - name: generated_answers, sequence: string: 字段generated_answers（生成答案列表），字符串序列 splits: - name: train, num_bytes: 35216238, num_examples: 28000: 训练集：数据量35216238字节，样本量28000 - name: validation, num_bytes: 2593667, num_examples: 2000: 验证集：数据量2593667字节，样本量2000 - name: test, num_bytes: 5095967, num_examples: 4000: 测试集：数据量5095967字节，样本量4000 download_size: 14752335: 总下载大小：14752335字节 dataset_size: 42905872: 数据集总大小：42905872字节 - config_name: default features: - name: id, dtype: string: 字段id，数据类型：字符串 - name: lang, dtype: string: 字段lang，数据类型：字符串 - name: question, dtype: string: 字段question（问题文本），数据类型：字符串 - name: answerText, dtype: string: 字段answerText（标准答案文本），数据类型：字符串 - name: category, dtype: string: 字段category（问题类别），数据类型：字符串 - name: complexityType, dtype: string: 字段complexityType（问题复杂度类型），数据类型：字符串 - name: questionEntity, list: 字段questionEntity（问题实体列表），包含子字段： - name: name, dtype: string: 实体名称，字符串类型 - name: entityType, dtype: string: 实体类型，字符串类型 - name: label, dtype: string: 实体标签，字符串类型 - name: mention, dtype: string: 实体文本提及，字符串类型 - name: span, list: int32: 实体在文本中的位置跨度，32位整数列表 - name: answerEntity, list: 字段answerEntity（答案实体列表），包含子字段： - name: name, dtype: string: 答案实体名称，字符串类型 - name: label, dtype: string: 答案实体标签，字符串类型 - name: relevant_triplets, sequence: sequence: string: 字段relevant_triplets（相关三元组列表），嵌套字符串序列 - name: verbalized_relevant_triplets, sequence: string: 字段verbalized_relevant_triplets（自然语言化相关三元组列表），字符串序列 - name: relevant_triplets_g2t, dtype: string: 字段relevant_triplets_g2t（文本生成相关三元组），字符串类型 splits: - name: train, num_bytes: 19478013, num_examples: 14000: 训练集：数据量19478013字节，样本量14000 - name: validation, num_bytes: 2791664, num_examples: 2000: 验证集：数据量2791664字节，样本量2000 - name: test, num_bytes: 5572329, num_examples: 4000: 测试集：数据量5572329字节，样本量4000 download_size: 8236687: 总下载大小：8236687字节 dataset_size: 27842006: 数据集总大小：27842006字节 configs: - config_name: T5-Large-SSM-answers data_files: - split: train, path: T5-Large-SSM-answers/train-*: 训练集数据路径：T5-Large-SSM-answers/train-* - split: validation, path: T5-Large-SSM-answers/validation-*: 验证集数据路径：T5-Large-SSM-answers/validation-* - split: test, path: T5-Large-SSM-answers/test-*: 测试集数据路径：T5-Large-SSM-answers/test-* - config_name: T5-Large-SSM-answers-linked data_files: - split: test, path: T5-Large-SSM-answers-linked/test-*: 测试集数据路径：T5-Large-SSM-answers-linked/test-* - config_name: T5-XL-SSM-answers data_files: - split: train, path: T5-XL-SSM-answers/train-*: 训练集数据路径：T5-XL-SSM-answers/train-* - split: validation, path: T5-XL-SSM-answers/validation-*: 验证集数据路径：T5-XL-SSM-answers/validation-* - split: test, path: T5-XL-SSM-answers/test-*: 测试集数据路径：T5-XL-SSM-answers/test-* - config_name: default data_files: - split: train, path: data/train-*: 训练集数据路径：data/train-* - split: validation, path: data/validation-*: 验证集数据路径：data/validation-* - split: test, path: data/test-*: 测试集数据路径：data/test-* # Mintaka：面向端到端问答的复杂自然多语言数据集 本数据集为原始Mintaka数据集的扩展版本，采用[KAPING](https://arxiv.org/abs/2306.04136v1)提出的方法，为问题中的实体提取相关三元组。 相关三元组已通过在WebNLG数据集上微调的[T5模型](https://huggingface.co/s-nlp/g2t-t5-xl-webnlg)转换为自然语言文本。 此外，本数据集还提供了在Mintaka数据集上微调的[T5 Large SSM](https://huggingface.co/msalnikov/kgqa-mintaka-t5-large-ssm)与[T5 XXL SSM](https://huggingface.co/msalnikov/kgqa-mintaka-t5-xl-ssm-nq)模型生成的答案，对应配置名称与上述模型匹配。
提供机构：
msalnikov
原始信息汇总

数据集概述

数据集名称： Mintaka

数据集任务：

  • 任务类别：question-answering
  • 任务ID：open-domain-qa

数据集语言：

  • 支持语言：ar, de, ja, hi, pt, en, es, it, fr
  • 语言BCP47代码：ar-SA, de-DE, ja-JP, hi-HI, pt-PT, en-EN, es-ES, it-IT, fr-FR

数据集许可证： cc-by-4.0

数据集大小：

  • 大小类别：100K<n<1M
  • 下载大小：9102570
  • 数据集总大小：24822107

数据集配置

配置名称： T5-Large-SSM-answers

  • 特征：
    • id: string
    • lang: string
    • question: string
    • answerText: string
    • category: string
    • complexityType: string
    • questionEntity:
      • entityType: string
      • label: string
      • mention: string
      • name: string
      • span: sequence: int32
    • answerEntity:
      • label: string
      • name: string
    • generated_answers: sequence: string
  • 分割：
    • train: 17007018 bytes, 14000 examples
    • validation: 3009249 bytes, 2000 examples
    • test: 4805840 bytes, 4000 examples

配置名称： T5-Large-SSM-answers-linked

  • 特征：
    • id: string
    • lang: string
    • question: string
    • answerText: string
    • category: string
    • complexityType: string
    • questionEntity:
      • entityType: string
      • label: string
      • mention: string
      • name: string
      • span: sequence: int32
    • answerEntity:
      • label: string
      • name: string
    • generated_answers: sequence: string
    • linked_generated_answers: sequence: sequence: string
  • 分割：
    • test: 7736223 bytes, 4000 examples

配置名称： T5-XL-SSM-answers

  • 特征：
    • id: string
    • lang: string
    • question: string
    • answerText: string
    • category: string
    • complexityType: string
    • questionEntity:
      • entityType: string
      • label: string
      • mention: string
      • name: string
      • span: sequence: int32
    • answerEntity:
      • label: string
      • name: string
    • generated_answers: sequence: string
  • 分割：
    • train: 35216238 bytes, 28000 examples
    • validation: 2593667 bytes, 2000 examples
    • test: 5095967 bytes, 4000 examples

配置名称： default

  • 特征：
    • id: string
    • lang: string
    • question: string
    • answerText: string
    • category: string
    • complexityType: string
    • questionEntity:
      • name: string
      • entityType: string
      • label: string
      • mention: string
      • span: list: int32
    • answerEntity:
      • name: string
      • label: string
    • relevant_triplets: sequence: sequence: string
    • verbalized_relevant_triplets: sequence: string
    • relevant_triplets_g2t: string
  • 分割：
    • train: 19478013 bytes, 14000 examples
    • validation: 2791664 bytes, 2000 examples
    • test: 5572329 bytes, 4000 examples
搜集汇总
数据集介绍
main_image_url
以上内容由遇见数据集搜集并总结生成
关于我们
遇见数据集——让每个数据集都被发现，让每一次遇见都有价值。
数发科官网www.sfktec.com
联系我们
youjian
selectdataset@iotsh.com.cn
商业合作
数据驱动未来，携手共赢发展
二维码
社区交流群
二维码
科研交流群
商业服务