mlqa
收藏魔搭社区2025-12-05 更新2025-05-24 收录
下载链接：
https://modelscope.cn/datasets/facebook/mlqa
下载链接
链接失效反馈官方服务：
资源简介：
# Dataset Card for "mlqa"
## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)
## Dataset Description
- **Homepage:** [https://github.com/facebookresearch/MLQA](https://github.com/facebookresearch/MLQA)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 4.15 GB
- **Size of the generated dataset:** 910.01 MB
- **Total amount of disk used:** 5.06 GB
### Dataset Summary
MLQA (MultiLingual Question Answering) is a benchmark dataset for evaluating cross-lingual question answering performance.
MLQA consists of over 5K extractive QA instances (12K in English) in SQuAD format in seven languages - English, Arabic,
German, Spanish, Hindi, Vietnamese and Simplified Chinese. MLQA is highly parallel, with QA instances parallel between
4 different languages on average.
### Supported Tasks and Leaderboards
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Languages
MLQA contains QA instances in 7 languages, English, Arabic, German, Spanish, Hindi, Vietnamese and Simplified Chinese.
## Dataset Structure
### Data Instances
#### mlqa-translate-test.ar
- **Size of downloaded dataset files:** 10.08 MB
- **Size of the generated dataset:** 5.48 MB
- **Total amount of disk used:** 15.56 MB
An example of 'test' looks as follows.
```
```
#### mlqa-translate-test.de
- **Size of downloaded dataset files:** 10.08 MB
- **Size of the generated dataset:** 3.88 MB
- **Total amount of disk used:** 13.96 MB
An example of 'test' looks as follows.
```
```
#### mlqa-translate-test.es
- **Size of downloaded dataset files:** 10.08 MB
- **Size of the generated dataset:** 3.92 MB
- **Total amount of disk used:** 13.99 MB
An example of 'test' looks as follows.
```
```
#### mlqa-translate-test.hi
- **Size of downloaded dataset files:** 10.08 MB
- **Size of the generated dataset:** 4.61 MB
- **Total amount of disk used:** 14.68 MB
An example of 'test' looks as follows.
```
```
#### mlqa-translate-test.vi
- **Size of downloaded dataset files:** 10.08 MB
- **Size of the generated dataset:** 6.00 MB
- **Total amount of disk used:** 16.07 MB
An example of 'test' looks as follows.
```
```
### Data Fields
The data fields are the same among all splits.
#### mlqa-translate-test.ar
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answers`: a dictionary feature containing:
- `answer_start`: a `int32` feature.
- `text`: a `string` feature.
- `id`: a `string` feature.
#### mlqa-translate-test.de
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answers`: a dictionary feature containing:
- `answer_start`: a `int32` feature.
- `text`: a `string` feature.
- `id`: a `string` feature.
#### mlqa-translate-test.es
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answers`: a dictionary feature containing:
- `answer_start`: a `int32` feature.
- `text`: a `string` feature.
- `id`: a `string` feature.
#### mlqa-translate-test.hi
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answers`: a dictionary feature containing:
- `answer_start`: a `int32` feature.
- `text`: a `string` feature.
- `id`: a `string` feature.
#### mlqa-translate-test.vi
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answers`: a dictionary feature containing:
- `answer_start`: a `int32` feature.
- `text`: a `string` feature.
- `id`: a `string` feature.
### Data Splits
| name |test|
|----------------------|---:|
|mlqa-translate-test.ar|5335|
|mlqa-translate-test.de|4517|
|mlqa-translate-test.es|5253|
|mlqa-translate-test.hi|4918|
|mlqa-translate-test.vi|5495|
## Dataset Creation
### Curation Rationale
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Source Data
#### Initial Data Collection and Normalization
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### Who are the source language producers?
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Annotations
#### Annotation process
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### Who are the annotators?
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Personal and Sensitive Information
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## Considerations for Using the Data
### Social Impact of Dataset
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Discussion of Biases
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Other Known Limitations
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## Additional Information
### Dataset Curators
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Licensing Information
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Citation Information
```
@article{lewis2019mlqa,
title = {MLQA: Evaluating Cross-lingual Extractive Question Answering},
author = {Lewis, Patrick and Oguz, Barlas and Rinott, Ruty and Riedel, Sebastian and Schwenk, Holger},
journal = {arXiv preprint arXiv:1910.07475},
year = 2019,
eid = {arXiv: 1910.07475}
}
```
### Contributions
Thanks to [@patrickvonplaten](https://github.com/patrickvonplaten), [@M-Salti](https://github.com/M-Salti), [@lewtun](https://github.com/lewtun), [@thomwolf](https://github.com/thomwolf), [@mariamabarham](https://github.com/mariamabarham), [@lhoestq](https://github.com/lhoestq) for adding this dataset.
# "MLQA"数据集卡片
## 目录
- [数据集描述](#dataset-description)
- [数据集概述](#dataset-summary)
- [支持任务与评测榜单](#supported-tasks-and-leaderboards)
- [语言覆盖](#languages)
- [数据集结构](#dataset-structure)
- [数据实例](#data-instances)
- [数据字段](#data-fields)
- [数据划分](#data-splits)
- [数据集构建](#dataset-creation)
- [构建初衷](#curation-rationale)
- [源数据](#source-data)
- [标注信息](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集维护者](#dataset-curators)
- [许可信息](#licensing-information)
- [引用信息](#citation-information)
- [贡献者](#contributions)
## 数据集描述
- **主页**：[https://github.com/facebookresearch/MLQA](https://github.com/facebookresearch/MLQA)
- **代码仓库**：[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **相关论文**：[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **联络人**：[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **下载数据集文件总大小**：4.15 GB
- **生成数据集总大小**：910.01 MB
- **磁盘总占用量**：5.06 GB
### 数据集概述
MLQA（多语言问答，MultiLingual Question Answering）是用于评估跨语言抽取式问答性能的基准数据集。MLQA包含7种语言的超5000条抽取式问答样本（英文样本超12000条），采用SQuAD格式，覆盖语言包括英语、阿拉伯语、德语、西班牙语、印地语、越南语及简体中文。MLQA具有高度的平行性，平均每条问答样本可在4种不同语言间实现平行对应。
### 支持任务与评测榜单
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 语言覆盖
MLQA包含7种语言的问答样本，分别为英语、阿拉伯语、德语、西班牙语、印地语、越南语及简体中文。
## 数据集结构
### 数据实例
#### mlqa-translate-test.ar
- **下载文件大小**：10.08 MB
- **生成数据集大小**：5.48 MB
- **磁盘总占用量**：15.56 MB
“测试集”的一个示例如下：
#### mlqa-translate-test.de
- **下载文件大小**：10.08 MB
- **生成数据集大小**：3.88 MB
- **磁盘总占用量**：13.96 MB
“测试集”的一个示例如下：
#### mlqa-translate-test.es
- **下载文件大小**：10.08 MB
- **生成数据集大小**：3.92 MB
- **磁盘总占用量**：13.99 MB
“测试集”的一个示例如下：
#### mlqa-translate-test.hi
- **下载文件大小**：10.08 MB
- **生成数据集大小**：4.61 MB
- **磁盘总占用量**：14.68 MB
“测试集”的一个示例如下：
#### mlqa-translate-test.vi
- **下载文件大小**：10.08 MB
- **生成数据集大小**：6.00 MB
- **磁盘总占用量**：16.07 MB
“测试集”的一个示例如下：
### 数据字段
所有划分的数据字段格式一致。
#### mlqa-translate-test.ar
- `context`：字符串（string）类型特征。
- `question`：字符串（string）类型特征。
- `answers`：字典类型特征，包含：
- `answer_start`：int32类型特征。
- `text`：字符串（string）类型特征。
- `id`：字符串（string）类型特征。
#### mlqa-translate-test.de
- `context`：字符串（string）类型特征。
- `question`：字符串（string）类型特征。
- `answers`：字典类型特征，包含：
- `answer_start`：int32类型特征。
- `text`：字符串（string）类型特征。
- `id`：字符串（string）类型特征。
#### mlqa-translate-test.es
- `context`：字符串（string）类型特征。
- `question`：字符串（string）类型特征。
- `answers`：字典类型特征，包含：
- `answer_start`：int32类型特征。
- `text`：字符串（string）类型特征。
- `id`：字符串（string）类型特征。
#### mlqa-translate-test.hi
- `context`：字符串（string）类型特征。
- `question`：字符串（string）类型特征。
- `answers`：字典类型特征，包含：
- `answer_start`：int32类型特征。
- `text`：字符串（string）类型特征。
- `id`：字符串（string）类型特征。
#### mlqa-translate-test.vi
- `context`：字符串（string）类型特征。
- `question`：字符串（string）类型特征。
- `answers`：字典类型特征，包含：
- `answer_start`：int32类型特征。
- `text`：字符串（string）类型特征。
- `id`：字符串（string）类型特征。
### 数据划分
| 数据集名称 | 测试集样本数 |
|----------------------|---:|
|mlqa-translate-test.ar|5335|
|mlqa-translate-test.de|4517|
|mlqa-translate-test.es|5253|
|mlqa-translate-test.hi|4918|
|mlqa-translate-test.vi|5495|
## 数据集构建
### 构建初衷
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 源数据
#### 初始数据收集与标准化
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### 源语言数据生产者是谁？
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 标注信息
#### 标注流程
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### 标注人员是谁？
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 个人与敏感信息
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## 数据集使用注意事项
### 数据集的社会影响
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 偏差讨论
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 其他已知局限
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## 附加信息
### 数据集维护者
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 许可信息
[更多信息请见](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 引用信息
@article{lewis2019mlqa,
title = {MLQA: 评估跨语言抽取式问答},
author = {Lewis, Patrick and Oguz, Barlas and Rinott, Ruty and Riedel, Sebastian and Schwenk, Holger},
journal = {arXiv预印本 arXiv:1910.07475},
year = 2019,
eid = {arXiv: 1910.07475}
}
### 贡献者
感谢[@patrickvonplaten](https://github.com/patrickvonplaten)、[@M-Salti](https://github.com/M-Salti)、[@lewtun](https://github.com/lewtun)、[@thomwolf](https://github.com/thomwolf)、[@mariamabarham](https://github.com/mariamabarham)、[@lhoestq](https://github.com/lhoestq) 贡献了该数据集。
提供机构：
maas
创建时间：
2025-05-20
