waboucay/turk_corpus

---
language:
- en
task_categories:
- text2text-generation
---

# Turk Corpus

<!-- Provide a quick summary of the dataset. -->

HuggingFace implementation of the Turk corpus for sentence simplification gathered by Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen and Chris Callison-Burch.

/!\ I am not one of the creators of the dataset, I just needed a HF version of this dataset and uploaded it. I encourage you to read the paper introducing the dataset: [Optimizing Statistical Machine Translation for Text Simplification](https://aclanthology.org/Q16-1029/) (2016)

<!-- ## Dataset Details

### Dataset Description -->

<!-- Provide a longer summary of what this dataset is. -->

<!-- - **Curated by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]

### Dataset Sources [optional] -->

<!-- Provide the basic links for the dataset. -->

<!-- - **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed] -->

## Uses

This dataset can be used to evaluate sentence simplification models.

<!-- ### Direct Use -->

<!-- This section describes suitable use cases for the dataset. -->

<!-- [More Information Needed]

### Out-of-Scope Use -->

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->

<!-- [More Information Needed] -->

## Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

- **Size of the generated dataset:** 2.4 MB

An example of 'test' looks as follows.
```
{
'complex': 'One side of the armed conflicts is composed mainly of the Sudanese military and the Janjaweed , a Sudanese militia group recruited mostly from the Afro-Arab Abbala tribes of the northern Rizeigat region in Sudan .',
'simple': [
'One side of the armed conflicts is made of Sudanese military and the Janjaweed , a Sudanese militia recruited from the Afro-Arab Abbala tribes of the northern Rizeigat region in Sudan .',
'One side of the armed conflicts is composed mainly of the Sudanese military and the Janjaweed, a Sudanese militia group recruited mostly from the Afro-Arab Abbala tribes of the northern Rizeigat regime in Sudan.',
'One side of the armed conflicts is made up mostly of the Sudanese military and the Janjaweed, a Sudanese militia group whose recruits mostly come from the Afro-Arab Abbala tribes from the northern Rizeigat region in Sudan.',
'One side of the armed conflicts is composed mainly of the Sudanese military and the Janjaweed , a Sudanese militia group recruited mostly from the Afro-Arab Abbala tribes in Sudan .',
'One side of the armed conflicts is composed mainly of the Sudanese military and the Janjaweed , a Sudanese militia group recruited mostly from the Afro-Arab Abbala tribes of the northern Rizeigat region in Sudan .',
'One side of the armed conflicts consist of the Sudanese military and the Sudanese militia group Janjaweed.',
'The Sudanese military and the Janjaweed make up one of the armed conflicts, mostly from the Afro-Arab Abbal tribes in Sudan.',
'One side of the armed conflicts is mainly Sudanese military and the Janjaweed, which recruited from the Afro-Arab Abbala tribes.'
]
}
```

<!-- ## Dataset Creation

### Curation Rationale

<!-- Motivation for the creation of this dataset. -->

<!-- [More Information Needed]

### Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->

<!-- #### Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->

<!-- [More Information Needed]

#### Who are the source data producers?

<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. -->

<!-- [More Information Needed]

### Annotations [optional]

<!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. -->

<!-- #### Annotation process

<!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. -->

<!-- [More Information Needed]

#### Who are the annotators?

<!-- This section describes the people or systems who created the annotations. -->

<!-- [More Information Needed]

#### Personal and Sensitive Information

<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. -->

<!-- [More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

<!-- [More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

<!-- Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations. -->

## Citation

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```
@article{xu-etal-2016-optimizing,
title = "Optimizing Statistical Machine Translation for Text Simplification",
author = "Xu, Wei and Napoles, Courtney and Pavlick, Ellie and Chen, Quanze and Callison-Burch, Chris",
editor = "Lee, Lillian and Johnson, Mark and Toutanova, Kristina",
journal = "Transactions of the Association for Computational Linguistics",
volume = "4",
year = "2016",
address = "Cambridge, MA",
publisher = "MIT Press",
url = "https://aclanthology.org/Q16-1029",
doi = "10.1162/tacl_a_00107",
pages = "401--415",
}
```

**ACL:**

Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing Statistical Machine Translation for Text Simplification. Transactions of the Association for Computational Linguistics, 4:401–415.

<!-- ## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. -->

<!-- [More Information Needed]

## More Information [optional]

[More Information Needed]

## Dataset Card Authors [optional]

[More Information Needed]

## Dataset Card Contact

[More Information Needed] -->

原始信息汇总

数据集概述

数据集名称

Turk Corpus

数据集简介

该数据集是HuggingFace实现的Turk corpus，用于句子简化任务。该数据集由Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen和Chris Callison-Burch收集。

数据集用途

该数据集主要用于评估句子简化模型。

数据集结构

• 数据集大小： 2.4 MB
• 数据示例： 包含一个复杂的句子及其多个简化版本。

引用信息

BibTeX

@article{xu-etal-2016-optimizing, title = "Optimizing Statistical Machine Translation for Text Simplification", author = "Xu, Wei and Napoles, Courtney and Pavlick, Ellie and Chen, Quanze and Callison-Burch, Chris", editor = "Lee, Lillian and Johnson, Mark and Toutanova, Kristina", journal = "Transactions of the Association for Computational Linguistics", volume = "4", year = "2016", address = "Cambridge, MA", publisher = "MIT Press", url = "https://aclanthology.org/Q16-1029", doi = "10.1162/tacl_a_00107", pages = "401--415", }

ACL

Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing Statistical Machine Translation for Text Simplification. Transactions of the Association for Computational Linguistics, 4:401–415.