multi_nli

# Dataset Card for Multi-Genre Natural Language Inference (MultiNLI)

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** [https://www.nyu.edu/projects/bowman/multinli/](https://www.nyu.edu/projects/bowman/multinli/)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 226.85 MB
- **Size of the generated dataset:** 76.95 MB
- **Total amount of disk used:** 303.81 MB

### Dataset Summary

The Multi-Genre Natural Language Inference (MultiNLI) corpus is a
crowd-sourced collection of 433k sentence pairs annotated with textual
entailment information. The corpus is modeled on the SNLI corpus, but differs in
that covers a range of genres of spoken and written text, and supports a
distinctive cross-genre generalization evaluation. The corpus served as the
basis for the shared task of the RepEval 2017 Workshop at EMNLP in Copenhagen.

### Supported Tasks and Leaderboards

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Languages

The dataset contains samples in English only.

## Dataset Structure

### Data Instances

- **Size of downloaded dataset files:** 226.85 MB
- **Size of the generated dataset:** 76.95 MB
- **Total amount of disk used:** 303.81 MB

Example of a data instance:

```
{
"promptID": 31193,
"pairID": "31193n",
"premise": "Conceptually cream skimming has two basic dimensions - product and geography.",
"premise_binary_parse": "( ( Conceptually ( cream skimming ) ) ( ( has ( ( ( two ( basic dimensions ) ) - ) ( ( product and ) geography ) ) ) . ) )",
"premise_parse": "(ROOT (S (NP (JJ Conceptually) (NN cream) (NN skimming)) (VP (VBZ has) (NP (NP (CD two) (JJ basic) (NNS dimensions)) (: -) (NP (NN product) (CC and) (NN geography)))) (. .)))",
"hypothesis": "Product and geography are what make cream skimming work. ",
"hypothesis_binary_parse": "( ( ( Product and ) geography ) ( ( are ( what ( make ( cream ( skimming work ) ) ) ) ) . ) )",
"hypothesis_parse": "(ROOT (S (NP (NN Product) (CC and) (NN geography)) (VP (VBP are) (SBAR (WHNP (WP what)) (S (VP (VBP make) (NP (NP (NN cream)) (VP (VBG skimming) (NP (NN work)))))))) (. .)))",
"genre": "government",
"label": 1
}
```

### Data Fields

The data fields are the same among all splits.

- `promptID`: Unique identifier for prompt
- `pairID`: Unique identifier for pair
- `{premise,hypothesis}`: combination of `premise` and `hypothesis`
- `{premise,hypothesis} parse`: Each sentence as parsed by the Stanford PCFG Parser 3.5.2
- `{premise,hypothesis} binary parse`: parses in unlabeled binary-branching format
- `genre`: a `string` feature.
- `label`: a classification label, with possible values including `entailment` (0), `neutral` (1), `contradiction` (2). Dataset instances which don't have any gold label are marked with -1 label. Make sure you filter them before starting the training using `datasets.Dataset.filter`.

### Data Splits

|train |validation_matched|validation_mismatched|
|-----:|-----------------:|--------------------:|
|392702| 9815| 9832|

## Dataset Creation

### Curation Rationale

They constructed MultiNLI so as to make it possible to explicitly evaluate models both on the quality of their sentence representations within the training domain and on their ability to derive reasonable representations in unfamiliar domains.

### Source Data

#### Initial Data Collection and Normalization

They created each sentence pair by selecting a premise sentence from a preexisting text source and asked a human annotator to compose a novel sentence to pair with it as a hypothesis.

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Licensing Information

The majority of the corpus is released under the OANC’s license, which allows all content to be freely used, modified, and shared under permissive terms. The data in the FICTION section falls under several permissive licenses; Seven Swords is available under a Creative Commons Share-Alike 3.0 Unported License, and with the explicit permission of the author, Living History and Password Incorrect are available under Creative Commons Attribution 3.0 Unported Licenses; the remaining works of fiction are in the public domain in the United States (but may be licensed differently elsewhere).

### Citation Information

```
@InProceedings{N18-1101,
author = "Williams, Adina
and Nangia, Nikita
and Bowman, Samuel",
title = "A Broad-Coverage Challenge Corpus for
Sentence Understanding through Inference",
booktitle = "Proceedings of the 2018 Conference of
the North American Chapter of the
Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long
Papers)",
year = "2018",
publisher = "Association for Computational Linguistics",
pages = "1112--1122",
location = "New Orleans, Louisiana",
url = "http://aclweb.org/anthology/N18-1101"
}
```

### Contributions

Thanks to [@bhavitvyamalik](https://github.com/bhavitvyamalik), [@patrickvonplaten](https://github.com/patrickvonplaten), [@thomwolf](https://github.com/thomwolf), [@mariamabarham](https://github.com/mariamabarham) for adding this dataset.

# 多体裁自然语言推理（Multi-Genre Natural Language Inference, MultiNLI）数据集卡片

## 目录
- [数据集描述](#dataset-description)
- [数据集摘要](#dataset-summary)
- [支持任务与排行榜](#supported-tasks-and-leaderboards)
- [语言](#languages)
- [数据集结构](#dataset-structure)
- [数据实例](#data-instances)
- [数据字段](#data-fields)
- [数据划分](#data-splits)
- [数据集构建](#dataset-creation)
- [构建初衷](#curation-rationale)
- [源数据](#source-data)
- [标注信息](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限性](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集维护者](#dataset-curators)
- [授权信息](#licensing-information)
- [引用信息](#citation-information)
- [贡献者](#contributions)

## 数据集描述

- **主页:** [https://www.nyu.edu/projects/bowman/multinli/](https://www.nyu.edu/projects/bowman/multinli/)
- **代码仓库:** [需补充更多信息](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **论文:** [需补充更多信息](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **联系人:** [需补充更多信息](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **下载数据集文件大小:** 226.85 MB
- **生成数据集大小:** 76.95 MB
- **总磁盘占用:** 303.81 MB

### 数据集摘要

多体裁自然语言推理（Multi-Genre Natural Language Inference, MultiNLI）语料库是一个包含43.3万个句子对的众包集合，所有样本均带有文本蕴含（textual entailment）标注。该语料库以SNLI语料库为原型构建，但相较后者，其覆盖了口语与书面文本的多种体裁，并支持独具特色的跨体裁泛化评估。该语料库曾作为2017年EMNLP会议哥本哈根分会RepEval研讨会共享任务的基准数据集。

### 支持任务与排行榜

[需补充更多信息](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 语言

本数据集仅包含英语样本。

## 数据集结构

### 数据实例

- **下载数据集文件大小:** 226.85 MB
- **生成数据集大小:** 76.95 MB
- **总磁盘占用:** 303.81 MB

数据实例示例：

json
{
"promptID": 31193,
"pairID": "31193n",
"premise": "Conceptually cream skimming has two basic dimensions - product and geography.",
"premise_binary_parse": "( ( Conceptually ( cream skimming ) ) ( ( has ( ( ( two ( basic dimensions ) ) - ) ( ( product and ) geography ) ) ) . ) )",
"premise_parse": "(ROOT (S (NP (JJ Conceptually) (NN cream) (NN skimming)) (VP (VBZ has) (NP (NP (CD two) (JJ basic) (NNS dimensions)) (: -) (NP (NN product) (CC and) (NN geography)))) (. .)))",
"hypothesis": "Product and geography are what make cream skimming work. ",
"hypothesis_binary_parse": "( ( ( Product and ) geography ) ( ( are ( what ( make ( cream ( skimming work ) ) ) ) ) . ) )",
"hypothesis_parse": "(ROOT (S (NP (NN Product) (CC and) (NN geography)) (VP (VBP are) (SBAR (WHNP (WP what)) (S (VP (VBP make) (NP (NP (NN cream)) (VP (VBG skimming) (NP (NN work)))))))) (. .)))",
"genre": "government",
"label": 1
}

### 数据字段

所有划分下的数据字段均保持一致。

- `promptID`: 前提的唯一标识符
- `pairID`: 句子对的唯一标识符
- `{premise,hypothesis}`: 前提（premise）与假设（hypothesis）文本
- `{premise,hypothesis} parse`: 经斯坦福PCFG句法分析器3.5.2（Stanford PCFG Parser 3.5.2）解析后的句子句法树
- `{premise,hypothesis} binary parse`: 采用无标签二元分支格式的句法解析结果
- `genre`: 字符串类型特征，表示文本体裁
- `label`: 分类标签，可选取值包括蕴含（entailment，对应0）、中性（neutral，对应1）、矛盾（contradiction，对应2）。无黄金标注的数据集实例将被标记为-1，请在训练前通过`datasets.Dataset.filter`方法过滤此类样本。

### 数据划分

|训练集|匹配验证集|不匹配验证集|
|-----:|-----------------:|--------------------:|
|392702| 9815| 9832|

## 数据集构建

### 构建初衷

团队构建MultiNLI的目的是，能够同时评估模型在训练域内的句子表征质量，以及在陌生域下生成合理表征的能力。

### 源数据

#### 初始数据收集与归一化

团队通过从已有文本源中选取前提句，并要求人工标注者创作一个全新的句子作为假设句，以此构建每一个句子对。

#### 源文本生产者为何人？

[需补充更多信息]

### 标注信息

#### 标注流程

[需补充更多信息]

#### 标注者为何人？

[需补充更多信息]

### 个人与敏感信息

[需补充更多信息]

## 数据集使用注意事项

### 数据集的社会影响

[需补充更多信息]

### 偏差讨论

[需补充更多信息]

### 其他已知局限性

[需补充更多信息]

## 附加信息

### 数据集维护者

[需补充更多信息](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 授权信息

本语料库的大部分内容遵循开放美国国家语料库（Open American National Corpus, OANC）的授权协议，允许所有内容在宽松条款下自由使用、修改与分享。其中FICTION（小说）板块的数据遵循多种宽松授权协议：《七剑》采用知识共享署名-相同方式共享3.0未本地化版本（Creative Commons Share-Alike 3.0 Unported）许可；经作者明确许可，《Living History》与《Password Incorrect》采用知识共享署名3.0未本地化版本许可；其余小说作品在美国境内属于公共领域（但在其他地区可能受不同授权条款约束）。

### 引用信息

bibtex
@InProceedings{N18-1101,
author = "Williams, Adina
and Nangia, Nikita
and Bowman, Samuel",
title = "A Broad-Coverage Challenge Corpus for
Sentence Understanding through Inference",
booktitle = "Proceedings of the 2018 Conference of
the North American Chapter of the
Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)",
year = "2018",
publisher = "Association for Computational Linguistics",
pages = "1112--1122",
location = "New Orleans, Louisiana",
url = "http://aclweb.org/anthology/N18-1101"
}

### 贡献者

感谢[@bhavitvyamalik](https://github.com/bhavitvyamalik)、[@patrickvonplaten](https://github.com/patrickvonplaten)、[@thomwolf](https://github.com/thomwolf)、[@mariamabarham](https://github.com/mariamabarham)为本数据集的添加工作。