KietReal/Vietnamese-English-translation

---
dataset_info:
features:
- name: English
dtype: string
- name: Vietnamese
dtype: string
- name: From
dtype: string
splits:
- name: train
num_bytes: 1557376339.6765032
num_examples: 6355063
- name: validation
num_bytes: 18379554.298004244
num_examples: 75000
- name: test
num_bytes: 6126518.0993347475
num_examples: 25000
download_size: 1119555973
dataset_size: 1581882412.073842
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
- split: test
path: data/test-*
license: mit
task_categories:
- translation
language:
- vi
- en
size_categories:
- 1M<n<10M
---
# Vietnamese-English Translation Dataset

## Overview

This dataset is a Vietnamese-English parallel corpus created by combining multiple high-quality machine translation datasets and synthetic data sources.

The dataset is designed for:

* Machine Translation (MT)
* Large Language Model (LLM) training
* Sequence-to-sequence learning
* Evaluation of Vietnamese ↔ English translation systems

---

## Data Sources

The dataset combines the following sources:

### 1. PhoMT

A large-scale Vietnamese-English translation dataset collected from news and parallel text resources.

Source:

* Vin/PhoMT

---

### 2. MTet

A Vietnamese-English translation dataset focused on conversational and general-domain translation tasks.

---

### 3. Synthetic Data generated by GPT-5.5 Pro (~10,000 samples)

Additional synthetic parallel data generated using GPT-5.5 Pro to increase:

* linguistic diversity
* conversational coverage
* paraphrasing variety
* low-frequency patterns

Synthetic data was manually cleaned and normalized before merging.

---

## Dataset Structure

Each sample contains:

```python
{
"Vietnamese": "...",
"English": "...",
"From": "PhoMT | MTet | GPT-5.5pro"
}
```

## Statistical Dataset

### Train set
📊 --- VIETNAMESE SENTENCE LENGTH STATISTICS ---
- Average: 24.7 words
- Maximum: 1827 words
- 90% of sentences are LESS THAN: 49 words
- 95% of sentences are LESS THAN: 68 words
- 99% of sentences are LESS THAN: 138 words

📊 --- ENGLISH SENTENCE LENGTH STATISTICS ---
- Average: 19.0 words
- Maximum: 1345 words
- 90% of sentences are LESS THAN: 37 words
- 95% of sentences are LESS THAN: 52 words
- 99% of sentences are LESS THAN: 105 words

### Validation set
📊 --- VIETNAMESE SENTENCE LENGTH STATISTICS ---
- Average: 24.9 words
- Maximum: 679 words
- 90% of sentences are LESS THAN: 50 words
- 95% of sentences are LESS THAN: 69 words
- 99% of sentences are LESS THAN: 141 words

📊 --- ENGLISH SENTENCE LENGTH STATISTICS ---
- Average: 19.1 words
- Maximum: 529 words
- 90% of sentences are LESS THAN: 38 words
- 95% of sentences are LESS THAN: 52 words
- 99% of sentences are LESS THAN: 107 words

### Test set
📊 --- VIETNAMESE SENTENCE LENGTH STATISTICS ---
- Average: 24.5 words
- Maximum: 479 words
- 90% of sentences are LESS THAN: 49 words
- 95% of sentences are LESS THAN: 67 words
- 99% of sentences are LESS THAN: 133 words

📊 --- ENGLISH SENTENCE LENGTH STATISTICS ---
- Average: 18.7 words
- Maximum: 388 words
- 90% of sentences are LESS THAN: 37 words
- 95% of sentences are LESS THAN: 51 words
- 99% of sentences are LESS THAN: 100 words

### Columns

| Column | Description |
| ---------- | ----------------------------- |
| Vietnamese | Vietnamese sentence |
| English | English sentence |
| From | Original source of the sample |

---

## Preprocessing

The dataset was cleaned using:

* removal of malformed punctuation
* whitespace normalization
* duplicate removal
* filtering empty rows
* normalization of dialogue symbols and special characters

Examples:

* remove repeated dashes
* normalize multiple spaces
* preserve valid hyphenated words such as `eco-friendly`

---

## Splits

The dataset is divided into:

* train
* validation
* test

with randomized shuffling before splitting.

---

## Intended Use

This dataset is intended for:

* Neural Machine Translation
* Transformer-based models
* Encoder-decoder architectures
* LLM fine-tuning
* Vietnamese NLP research

---

## Limitations

* Synthetic data may contain occasional translation artifacts.
* Some domains may be overrepresented depending on the source distribution.
* Automatic cleaning may imperfectly normalize punctuation in rare cases.

---

## License

Please refer to the original datasets:

* PhoMT
* MTet

Synthetic data generated by GPT-5.5 Pro follows OpenAI usage policies.

---

## Citation

If you use this dataset, please cite:

* PhoMT
* MTet
* this repository

---

## Author

Created by:

* KietReal

## 📬 Contact
* If you have any questions, feel free to contact me on my LinkedIn.
* For questions or improvements, feel free to open an issue or contribute.