PolyAgent/wiki_uk_en_parallel
收藏Hugging Face2024-08-11 更新2025-07-05 收录
下载链接：
https://hf-mirror.com/datasets/PolyAgent/wiki_uk_en_parallel
下载链接
链接失效反馈官方服务：
资源简介：
---
language:
- uk
- en
dataset_info:
features:
- name: id
dtype: string
- name: url
dtype: string
- name: title
dtype: string
- name: text
dtype: string
splits:
- name: train
num_bytes: 9744003868
num_examples: 2119045
- name: test
num_bytes: 3764150
num_examples: 1780
download_size: 5366910719
dataset_size: 9747768018
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
---
Ukrainian dump of the Wikipedia from June 20th 2024, plus equivalent articles from the English Wikipedia same date (turned out similar amount of tokens). Validation is reduced to <0.1% of the dataset for faster evaluation and avoiding losing valuable training data
v2 + spaces of different types and newlines replaced by a single normal space or newline.
v3, but removing stress marks such the ones in Олекса́ндр, as received feedback that they are unwanted
提供机构：
PolyAgent
