euconst

# Dataset Card for OPUS EUconst

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://opus.nlpl.eu/EUconst/corpus/version/EUconst
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Leaderboard:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Dataset Summary

A parallel corpus collected from the European Constitution.

EUconst's Numbers:
- Languages: 21
- Bitexts: 210
- Number of files: 986
- Number of tokens: 3.01M
- Sentence fragments: 0.22M

### Supported Tasks and Leaderboards

The underlying task is machine translation.

### Languages

The languages in the dataset are:
- Czech (`cs`)
- Danish (`da`)
- German (`de`)
- Greek (`el`)
- English (`en`)
- Spanish (`es`)
- Estonian (`et`)
- Finnish (`fi`)
- French (`fr`)
- Irish (`ga`)
- Hungarian (`hu`)
- Italian (`it`)
- Lithuanian (`lt`)
- Latvian (`lv`)
- Maltese (`mt`)
- Dutch (`nl`)
- Polish (`pl`)
- Portuguese (`pt`)
- Slovak (`sk`)
- Slovenian (`sl`)
- Swedish (`sv`)

## Dataset Structure

### Data Instances

```
{
"translation": {
"cs": "Celex Test ",
"da": "Celex Test "
}
}
```

### Data Fields

- `translation` (`dict`): Parallel sentences for the pair of languages.

### Data Splits

The dataset contains a single "train" split for each language pair.

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

If you use any part of the corpus in your own work, please cite the following article:
```
@inproceedings{tiedemann-2012-parallel,
title = "Parallel Data, Tools and Interfaces in {OPUS}",
author = {Tiedemann, J{\"o}rg},
editor = "Calzolari, Nicoletta and
Choukri, Khalid and
Declerck, Thierry and
Do{\u{g}}an, Mehmet U{\u{g}}ur and
Maegaard, Bente and
Mariani, Joseph and
Moreno, Asuncion and
Odijk, Jan and
Piperidis, Stelios",
booktitle = "Proceedings of the Eighth International Conference on Language Resources and Evaluation ({LREC}'12)",
month = may,
year = "2012",
address = "Istanbul, Turkey",
publisher = "European Language Resources Association (ELRA)",
url = "http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf",
pages = "2214--2218",
abstract = "This paper presents the current status of OPUS, a growing language resource of parallel corpora and related tools. The focus in OPUS is to provide freely available data sets in various formats together with basic annotation to be useful for applications in computational linguistics, translation studies and cross-linguistic corpus studies. In this paper, we report about new data sets and their features, additional annotation tools and models provided from the website and essential interfaces and on-line services included in the project.",
}
```

### Contributions

Thanks to [@patil-suraj](https://github.com/patil-suraj) for adding this dataset.

# OPUS EUconst 数据集卡片

## 目录
- [数据集描述](#dataset-description)
- [数据集概述](#dataset-summary)
- [支持的任务与排行榜](#supported-tasks-and-leaderboards)
- [语言范围](#languages)
- [数据集结构](#dataset-structure)
- [数据样例](#data-instances)
- [数据字段](#data-fields)
- [数据集划分](#data-splits)
- [数据集构建](#dataset-creation)
- [构建初衷](#curation-rationale)
- [源数据](#source-data)
- [标注信息](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限性](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集维护者](#dataset-curators)
- [许可信息](#licensing-information)
- [引用信息](#citation-information)
- [贡献致谢](#contributions)

## 数据集描述

- **主页：** https://opus.nlpl.eu/EUconst/corpus/version/EUconst
- **代码仓库：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **相关论文：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **排行榜：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **联系方式：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 数据集概述

本数据集为从《欧洲宪法》中采集的平行语料库。

EUconst 数据集统计数据：
- 涵盖语言：21种
- 双语平行句对：210组
- 文件总数：986个
- Token 总数：301万
- 语句片段数：22万

### 支持的任务与排行榜

其核心任务为机器翻译。

### 语言范围

数据集包含以下语言：
- 捷克语（`cs`）
- 丹麦语（`da`）
- 德语（`de`）
- 希腊语（`el`）
- 英语（`en`）
- 西班牙语（`es`）
- 爱沙尼亚语（`et`）
- 芬兰语（`fi`）
- 法语（`fr`）
- 爱尔兰语（`ga`）
- 匈牙利语（`hu`）
- 意大利语（`it`）
- 立陶宛语（`lt`）
- 拉脱维亚语（`lv`）
- 马耳他语（`mt`）
- 荷兰语（`nl`）
- 波兰语（`pl`）
- 葡萄牙语（`pt`）
- 斯洛伐克语（`sk`）
- 斯洛文尼亚语（`sl`）
- 瑞典语（`sv`）

## 数据集结构

### 数据样例

{
"translation": {
"cs": "Celex 测试用例 ",
"da": "Celex 测试用例 "
}
}

### 数据字段

- `translation`（字典类型）：对应语言对的平行语句。

### 数据集划分

该数据集为每个语言对仅包含一个「训练（train）」划分。

## 数据集构建

### 构建初衷

[更多信息待补充]

### 源数据

#### 初始数据采集与标准化

[更多信息待补充]

#### 源语言生成者是谁？

[更多信息待补充]

### 标注信息

#### 标注流程

[更多信息待补充]

#### 标注人员是谁？

[更多信息待补充]

### 个人与敏感信息

[更多信息待补充]

## 数据集使用注意事项

### 数据集的社会影响

[更多信息待补充]

### 偏差讨论

[更多信息待补充]

### 其他已知局限性

[更多信息待补充]

## 附加信息

### 数据集维护者

[更多信息待补充]

### 许可信息

[更多信息待补充]

### 引用信息

若您在研究中使用该语料库的任意部分，请引用以下文献：

@inproceedings{tiedemann-2012-parallel,
title = "Parallel Data, Tools and Interfaces in {OPUS}",
author = {Tiedemann, J{"o}rg},
editor = "Calzolari, Nicoletta and
Choukri, Khalid and
Declerck, Thierry and
Do{"u}g}an, Mehmet U{"u}gur and
Maegaard, Bente and
Mariani, Joseph and
Moreno, Asuncion and
Odijk, Jan and
Piperidis, Stelios",
booktitle = "Proceedings of the Eighth International Conference on Language Resources and Evaluation ({LREC}'12)",
month = may,
year = "2012",
address = "Istanbul, Turkey",
publisher = "European Language Resources Association (ELRA)",
url = "http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf",
pages = "2214--2218",
abstract = "This paper presents the current status of OPUS, a growing language resource of parallel corpora and related tools. The focus in OPUS is to provide freely available data sets in various formats together with basic annotation to be useful for applications in computational linguistics, translation studies and cross-linguistic corpus studies. In this paper, we report about new data sets and their features, additional annotation tools and models provided from the website and essential interfaces and on-line services included in the project.",
}

### 贡献致谢

感谢 [@patil-suraj](https://github.com/patil-suraj) 为本数据集提供的收录工作。