dialogsum_reformat

# Dataset Card for DIALOGSum Corpus
## Dataset Description
### Links
- **Homepage:** https://aclanthology.org/2021.findings-acl.449
- **Repository:** https://github.com/cylnlp/dialogsum
- **Paper:** https://aclanthology.org/2021.findings-acl.449
- **Point of Contact:** https://huggingface.co/knkarthick

### Dataset Summary
DialogSum is a large-scale dialogue summarization dataset, consisting of 13,460 (Plus 100 holdout data for topic generation) dialogues with corresponding manually labeled summaries and topics.
### Languages
English

## Dataset Structure
### Data Instances
DialogSum is a large-scale dialogue summarization dataset, consisting of 13,460 dialogues (+1000 tests) split into train, test and validation.
The first instance in the training set:
{'id': 'train_0', 'summary': "Mr. Smith's getting a check-up, and Doctor Hawkins advises him to have one every year. Hawkins'll give some information about their classes and medications to help Mr. Smith quit smoking.", 'dialogue': "#Person1#: Hi, Mr. Smith. I'm Doctor Hawkins. Why are you here today?\n#Person2#: I found it would be a good idea to get a check-up.\n#Person1#: Yes, well, you haven't had one for 5 years. You should have one every year.\n#Person2#: I know. I figure as long as there is nothing wrong, why go see the doctor?\n#Person1#: Well, the best way to avoid serious illnesses is to find out about them early. So try to come at least once a year for your own good.\n#Person2#: Ok.\n#Person1#: Let me see here. Your eyes and ears look fine. Take a deep breath, please. Do you smoke, Mr. Smith?\n#Person2#: Yes.\n#Person1#: Smoking is the leading cause of lung cancer and heart disease, you know. You really should quit.\n#Person2#: I've tried hundreds of times, but I just can't seem to kick the habit.\n#Person1#: Well, we have classes and some medications that might help. I'll give you more information before you leave.\n#Person2#: Ok, thanks doctor.", 'topic': "get a check-up}
### Data Fields
- dialogue: text of dialogue.
- summary: human written summary of the dialogue.
- topic: human written topic/one liner of the dialogue.
- id: unique file id of an example.

### Data Splits
- train: 12460
- val: 500
- test: 1500
- holdout: 100 [Only 3 features: id, dialogue, topic]

## Dataset Creation
### Curation Rationale
In paper:
We collect dialogue data for DialogSum from three public dialogue corpora, namely Dailydialog (Li et al., 2017), DREAM (Sun et al., 2019) and MuTual (Cui et al., 2019), as well as an English speaking practice website. These datasets contain face-to-face spoken dialogues that cover a wide range of daily-life topics, including schooling, work, medication, shopping, leisure, travel. Most conversations take place between friends, colleagues, and between service providers and customers.

Compared with previous datasets, dialogues from DialogSum have distinct characteristics:

Under rich real-life scenarios, including more diverse task-oriented scenarios;
Have clear communication patterns and intents, which is valuable to serve as summarization sources;
Have a reasonable length, which comforts the purpose of automatic summarization.

We ask annotators to summarize each dialogue based on the following criteria:
Convey the most salient information;
Be brief;
Preserve important named entities within the conversation;
Be written from an observer perspective;
Be written in formal language.
### Who are the source language producers?
linguists
### Who are the annotators?
language experts

## Licensing Information
non-commercial licence: MIT
## Citation Information
```
@inproceedings{chen-etal-2021-dialogsum,
title = "{D}ialog{S}um: {A} Real-Life Scenario Dialogue Summarization Dataset",
author = "Chen, Yulong and
Liu, Yang and
Chen, Liang and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.findings-acl.449",
doi = "10.18653/v1/2021.findings-acl.449",
pages = "5062--5074",
```
## Contributions
Thanks to [@cylnlp](https://github.com/cylnlp) for adding this dataset.

# DIALOGSum语料库数据集卡片
## 数据集说明
### 链接
- **主页:** https://aclanthology.org/2021.findings-acl.449
- **代码仓库:** https://github.com/cylnlp/dialogsum
- **相关论文:** https://aclanthology.org/2021.findings-acl.449
- **联系方式:** https://huggingface.co/knkarthick

### 数据集概览
DIALOGSum是一款大规模对话摘要数据集（dialogue summarization dataset），包含13460段对话（另有100条用于主题生成的预留数据），配套带有人工标注的摘要与主题标签。

### 语言
英语

## 数据集结构
### 数据样例
DIALOGSum是一款大规模对话摘要数据集，包含13460段对话（+1000条测试数据），划分为训练集、测试集与验证集。
训练集的第一条样例如下：

{'id': 'train_0', 'summary': "Mr. Smith's getting a check-up, and Doctor Hawkins advises him to have one every year. Hawkins'll give some information about their classes and medications to help Mr. Smith quit smoking.", 'dialogue': "#Person1#: Hi, Mr. Smith. I'm Doctor Hawkins. Why are you here today?
#Person2#: I found it would be a good idea to get a check-up.
#Person1#: Yes, well, you haven't had one for 5 years. You should have one every year.
#Person2#: I know. I figure as long as there is nothing wrong, why go see the doctor?
#Person1#: Well, the best way to avoid serious illnesses is to find out about them early. So try to come at least once a year for your own good.
#Person2#: Ok.
#Person1#: Let me see here. Your eyes and ears look fine. Take a deep breath, please. Do you smoke, Mr. Smith?
#Person2#: Yes.
#Person1#: Smoking is the leading cause of lung cancer and heart disease, you know. You really should quit.
#Person2#: I've tried hundreds of times, but I just can't seem to kick the habit.
#Person1#: Well, we have classes and some medications that might help. I'll give you more information before you leave.
#Person2#: Ok, thanks doctor.", 'topic': "get a check-up}

### 数据字段
- dialogue：对话文本
- summary：人工撰写的对话摘要
- topic：人工撰写的对话主题/单行概要
- id：单条样例的唯一文件标识符

### 数据划分
- 训练集（train）：12460条
- 验证集（val）：500条
- 测试集（test）：1500条
- 预留集（holdout set）：100条，仅包含id、dialogue、topic三个字段

## 数据集构建
### 数据遴选依据
论文中提及：
我们从三个公开对话语料库，即Dailydialog（Li等，2017）、DREAM（Sun等，2019）与MuTual（Cui等，2019），以及一个英语口语练习网站采集了DIALOGSum所需的对话数据。上述语料库均包含面对面口语对话，涵盖学业、工作、医疗、购物、休闲、出行等诸多日常话题，多数对话发生于朋友、同事之间，或是服务提供者与客户之间。

与过往数据集相比，DIALOGSum的对话具备如下鲜明特征：
1. 覆盖丰富的真实生活场景，包含更多样化的任务导向型场景；
2. 具备清晰的沟通模式与意图，可作为优质的摘要生成源；
3. 对话长度适中，适配自动摘要任务的需求。

我们要求标注人员基于以下准则生成对话摘要：
- 传达最核心的关键信息；
- 表述简洁凝练；
- 保留对话中的重要命名实体；
- 以旁观者视角撰写；
- 使用正式书面语。

### 语料来源生产者
语言学家

### 标注人员
语言领域专家

## 许可证信息
非商业性许可证：MIT

## 引用信息

@inproceedings{chen-etal-2021-dialogsum,
title = "{D}ialog{S}um: {A} Real-Life Scenario Dialogue Summarization Dataset",
author = "Chen, Yulong and
Liu, Yang and
Chen, Liang and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.findings-acl.449",
doi = "10.18653/v1/2021.findings-acl.449",
pages = "5062--5074",

## 致谢
感谢[@cylnlp](https://github.com/cylnlp)贡献本数据集。