natgillin/n-way-translations

这是一个多语言N-way平行语料库，专门用于机器翻译任务。数据集的每一行代表一个逻辑句子，该句子在上游源数据中可用多种语言表达，形成N-way并行结构，意味着每个句子在多种语言中都有对应翻译。数据集包含一个配置tatoeba-flatten，源自alvations/tatoeba-flatten，共有215,912行数据，支持最多50种语言，包括英语、德语、西班牙语、韩语、荷兰语、葡萄牙语、日语、意大利语、波兰语、法语、阿拉伯语、乌克兰语、俄语、希伯来语、匈牙利语、希腊语、保加利亚语、瑞典语、克罗地亚语、土耳其语、越南语、孟加拉语、塞尔维亚语、斯洛伐克语、芬兰语、捷克语、白俄罗斯语、印地语、罗马尼亚语、丹麦语、挪威语、乌兹别克语、冰岛语、印尼语、加泰罗尼亚语、阿塞拜疆语、马来语、泰语、乌尔都语、旁遮普语、爱沙尼亚语、马拉地语、泰米尔语、古吉拉特语、斯洛文尼亚语、马拉雅拉姆语、泰卢固语、卡纳达语、奥里亚语和中文。数据模式包括sentence_id（句子标识符）、origin（上游源标识符）、eng（英语文本，如果存在）和translations（一个列表，包含每种非空语言的lang和text字段）。数据集的设计便于过滤、排序和自包含访问，适用于多语言机器翻译模型训练和研究。This is a multilingual N-way parallel corpus designed for machine translation tasks. Each row in the dataset represents one logical sentence expressed in as many languages as available from the upstream source, forming an N-way parallel structure where each sentence has corresponding translations in multiple languages. The dataset includes a configuration named tatoeba-flatten, sourced from alvations/tatoeba-flatten, with 215,912 rows and support for up to 50 languages, including English, German, Spanish, Korean, Dutch, Portuguese, Japanese, Italian, Polish, French, Arabic, Ukrainian, Russian, Hebrew, Hungarian, Greek, Bulgarian, Swedish, Croatian, Turkish, Vietnamese, Bengali, Serbian, Slovak, Finnish, Czech, Belarusian, Hindi, Romanian, Danish, Norwegian, Uzbek, Icelandic, Indonesian, Catalan, Azerbaijani, Malay, Thai, Urdu, Punjabi, Estonian, Marathi, Tamil, Gujarati, Slovenian, Malayalam, Telugu, Kannada, Oriya, and Chinese. The schema consists of sentence_id (a unique identifier), origin (upstream source identifier), eng (English text, if present), and translations (a list of structs with lang and text fields for each non-null language). The dataset is structured for easy filtering, sorting, and self-contained access, making it suitable for training and research in multilingual machine translation models.