有没有相关的论文或文献参考？
这个数据集是基于什么背景创建的？
数据集的作者是谁？
能帮我联系到这个数据集的作者吗？
这个数据集如何下载？
IWSLT/iwslt2017|机器翻译数据集|多语种数据集
收藏hugging_face2023-04-05 更新2024-05-25 收录
下载链接：
https://hf-mirror.com/datasets/IWSLT/iwslt2017
下载链接
链接失效反馈资源简介：
IWSLT 2017数据集是一个多语言翻译数据集，涵盖了多种语言对，包括英语、阿拉伯语、德语、荷兰语、意大利语、罗马尼亚语、法语、日语、韩语和中文。数据集的主要任务是文本翻译，包括零样本翻译。数据集的结构包括训练集、验证集和测试集，每个语言对都有相应的数据实例和字段。数据集的创建过程、注释过程以及使用数据时的考虑因素等信息未在README中详细描述。
提供机构：
IWSLT
原始信息汇总
数据集概述
基本信息
- 数据集名称: IWSLT 2017
- 数据集大小: 1M<n<10M
- 语言: 阿拉伯语 (ar), 德语 (de), 英语 (en), 法语 (fr), 意大利语 (it), 日语 (ja), 韩语 (ko), 荷兰语 (nl), 罗马尼亚语 (ro), 中文 (zh)
- 语言创建方式: 专家生成
- 许可证: cc-by-nc-nd-4.0
- 多语言性: 翻译
数据集配置
- 配置名称: iwslt2017-en-it, iwslt2017-en-nl, iwslt2017-en-ro, iwslt2017-it-en, iwslt2017-it-nl, iwslt2017-it-ro, iwslt2017-nl-en, iwslt2017-nl-it, iwslt2017-nl-ro, iwslt2017-ro-en, iwslt2017-ro-it, iwslt2017-ro-nl, iwslt2017-ar-en, iwslt2017-de-en, iwslt2017-en-ar, iwslt2017-en-de, iwslt2017-en-fr, iwslt2017-en-ja, iwslt2017-en-ko, iwslt2017-en-zh, iwslt2017-fr-en, iwslt2017-ja-en, iwslt2017-ko-en, iwslt2017-zh-en
- 特征: 翻译
- 数据分割: 训练, 测试, 验证
详细数据分割
| 配置名称 | 分割 | 示例数量 | 字节数 |
|---|---|---|---|
| iwslt2017-en-it | 训练 | 231619 | 46647925 |
| iwslt2017-en-it | 测试 | 1566 | 305246 |
| iwslt2017-en-it | 验证 | 929 | 200023 |
| iwslt2017-en-nl | 训练 | 237240 | 42843933 |
| iwslt2017-en-nl | 测试 | 1777 | 311646 |
| iwslt2017-en-nl | 验证 | 1003 | 197814 |
| iwslt2017-en-ro | 训练 | 220538 | 44129950 |
| iwslt2017-en-ro | 测试 | 1678 | 316790 |
| iwslt2017-en-ro | 验证 | 914 | 205028 |
| iwslt2017-it-en | 训练 | 231619 | 46647925 |
| iwslt2017-it-en | 测试 | 1566 | 305246 |
| iwslt2017-it-en | 验证 | 929 | 200023 |
| iwslt2017-it-nl | 训练 | 233415 | 43033168 |
| iwslt2017-it-nl | 测试 | 1669 | 309725 |
| iwslt2017-it-nl | 验证 | 1001 | 197774 |
| iwslt2017-it-ro | 训练 | 217551 | 44485169 |
| iwslt2017-it-ro | 测试 | 1643 | 314974 |
| iwslt2017-it-ro | 验证 | 914 | 204989 |
| iwslt2017-nl-en | 训练 | 237240 | 42843933 |
| iwslt2017-nl-en | 测试 | 1777 | 311646 |
| iwslt2017-nl-en | 验证 | 1003 | 197814 |
| iwslt2017-nl-it | 训练 | 233415 | 43033168 |
| iwslt2017-nl-it | 测试 | 1669 | 309725 |
| iwslt2017-nl-it | 验证 | 1001 | 197774 |
| iwslt2017-nl-ro | 训练 | 206920 | 41338738 |
| iwslt2017-nl-ro | 测试 | 1680 | 320952 |
| iwslt2017-nl-ro | 验证 | 913 | 202380 |
| iwslt2017-ro-en | 训练 | 220538 | 44129950 |
| iwslt2017-ro-en | 测试 | 1678 | 316790 |
| iwslt2017-ro-en | 验证 | 914 | 205028 |
| iwslt2017-ro-it | 训练 | 217551 | 44485169 |
| iwslt2017-ro-it | 测试 | 1643 | 314974 |
| iwslt2017-ro-it | 验证 | 914 | 204989 |
| iwslt2017-ro-nl | 训练 | 206920 | 41338738 |
| iwslt2017-ro-nl | 测试 | 1680 | 320952 |
| iwslt2017-ro-nl | 验证 | 913 | 202380 |
| iwslt2017-ar-en | 训练 | 231713 | 56481059 |
| iwslt2017-ar-en | 测试 | 8583 | 2014296 |
| iwslt2017-ar-en | 验证 | 888 | 241206 |
| iwslt2017-de-en | 训练 | 206112 | 42608380 |
| iwslt2017-de-en | 测试 | 8079 | 1608474 |
| iwslt2017-de-en | 验证 | 888 | 210975 |
| iwslt2017-en-ar | 训练 | 231713 | 56481059 |
| iwslt2017-en-ar | 测试 | 8583 | 2014296 |
| iwslt2017-en-ar | 验证 | 888 | 241206 |
| iwslt2017-en-de | 训练 | 206112 | 42608380 |
| iwslt2017-en-de | 测试 | 8079 | 1608474 |
| iwslt2017-en-de | 验证 | 888 | 210975 |
| iwslt2017-en-fr | 训练 | 232825 | 49273286 |
| iwslt2017-en-fr | 测试 | 8597 | 1767465 |
| iwslt2017-en-fr | 验证 | 890 | 207579 |
| iwslt2017-en-ja | 训练 | 223108 | 48204987 |
| iwslt2017-en-ja | 测试 | 8469 | 1809007 |
| iwslt2017-en-ja | 验证 | 871 | 208124 |
| iwslt2017-en-ko | 训练 | 230240 | 51678043 |
| iwslt2017-en-ko | 测试 | 8514 | 1869793 |
| iwslt2017-en-ko | 验证 | 879 | 219295 |
| iwslt2017-en-zh | 训练 | 231266 | 44271004 |
| iwslt2017-en-zh | 测试 | 8549 | 1605527 |
| iwslt2017-en-zh | 验证 | 879 | 202537 |
| iwslt2017-fr-en | 训练 | 232825 | 49273286 |
| iwslt2017-fr-en | 测试 | 8597 | 1767465 |
| iwslt2017-fr-en | 验证 | 890 | 207579 |
| iwslt2017-ja-en | 训练 | 223108 | 48204987 |
| iwslt2017-ja-en | 测试 | 8469 | 1809007 |
| iwslt2017-ja-en | 验证 | 871 | 208124 |
| iwslt2017-ko-en | 训练 | 230240 | 51678043 |
| iwslt2017-ko-en | 测试 | 8514 | 1869793 |
| iwslt2017-ko-en | 验证 | 879 | 219295 |
| iwslt2017-zh-en | 训练 | 231266 | 44271004 |
| iwslt2017-zh-en | 测试 | 8549 | 1605527 |
| iwslt2017-zh-en | 验证 | 879 | 202537 |
数据集特征
- 特征名称: 翻译
- 数据类型: 多语言字符串
- 支持语言: 根据配置不同，支持多种语言组合，如英语-意大利语, 英语-荷兰语等。
数据集分割
- 训练集: 用于模型训练的数据集，包含大量样本。
- 测试集: 用于评估模型性能的数据集，通常包含一定数量的样本。
- 验证集: 用于调整模型参数和超参数的数据集，帮助优化模型性能。
AI搜集汇总
数据集介绍
构建方式
IWSLT 2017数据集的构建基于多种语言的翻译任务，包括零样本翻译。数据集包含了英语、德语、荷兰语、意大利语和罗马尼亚语等多种语言的翻译，以及作为非官方任务的英语与阿拉伯语、法语、日语、中文、德语和韩语的双语翻译。数据来源于TED演讲的转录文本，经过专家和众包的注释，形成了具有训练、验证和测试三个部分的数据集。
使用方法
使用IWSLT 2017数据集时，用户可以根据需要选择不同的语言对进行翻译训练。数据集提供了训练、验证和测试三个部分，可以用于模型的训练和评估。用户可以通过HuggingFace的datasets库来加载和利用这个数据集。
背景与挑战
背景概述
IWSLT 2017数据集是由国际口语翻译工作组（International Workshop on Spoken Language Translation）创建的，旨在促进口语翻译技术的发展。该数据集于2017年发布，由Mauro Cettolo等研究人员主导，包含了多种语言的翻译数据，主要涵盖了英语与其他语言（如德语、荷兰语、意大利语、罗马尼亚语、阿拉伯语、法语、日语、韩语和中文）之间的双向翻译。IWSLT 2017数据集的创建，是为了解决口语翻译领域中的实际问题，并为相关研究提供高质量的翻译数据，它在自然语言处理和机器翻译领域具有较高的影响力。
当前挑战
在构建IWSLT 2017数据集的过程中，研究人员面临着多方面的挑战。首先，多语言翻译数据的收集和整理工作繁重，需要确保数据的准确性和多样性。其次，数据集的构建需要考虑到不同语言之间的对应关系，尤其是在零样本翻译（zero-shot translation）的情境下。此外，数据集在收集和标注过程中，还要确保不包含个人敏感信息，避免引发隐私问题。在使用该数据集时，研究人员还需关注数据中可能存在的偏见和局限性，以确保研究结果的公正性和可靠性。
常用场景
经典使用场景
IWSLT 2017数据集是国际口语翻译领域的重要资源，其经典使用场景主要集中于机器翻译模型的训练与评估。该数据集支持多语言翻译任务，包括零样本翻译，为研究者提供了一个统一的平台来测试和比较不同翻译系统的性能。
解决学术问题
该数据集解决了机器翻译领域中的多个学术问题，如跨语言信息传递的准确性、翻译系统的鲁棒性以及不同语言对之间翻译的效率。通过提供多种语言对的平行语料，IWSLT 2017助力研究者探索和解决这些挑战，推动了翻译技术的发展。
实际应用
在实际应用中，IWSLT 2017数据集被广泛应用于机器翻译服务、语言学习工具以及跨文化交流平台。它为这些应用提供了高质量的训练数据，从而使得翻译服务更加精准，语言学习工具更加智能，跨文化交流更加顺畅。
数据集最近研究
最新研究方向
IWSLT 2017数据集最新研究方向主要聚焦于文本翻译，尤其是零样本翻译，旨在通过单一机器翻译系统实现包括英语、德语、荷兰语、意大利语和罗马尼亚语在内的多语言翻译任务。此外，该数据集还支持英语与其他语言（如阿拉伯语、法语、日语、中文、德语和韩语）之间的传统双向文本翻译研究。当前研究的热点事件包括提升翻译模型的准确性和效率，以及探索跨语言信息处理的新方法。这些研究对于推动机器翻译技术的发展和应用具有重要的意义。
以上内容由AI搜集并总结生成
数据主题
5,000+
优质数据集
54 个
任务类型
进入经典数据集
热门数据集
Google Scholar
Google Scholar是一个学术搜索引擎，旨在检索学术文献、论文、书籍、摘要和文章等。它涵盖了广泛的学科领域，包括自然科学、社会科学、艺术和人文学科。用户可以通过关键词搜索、作者姓名、出版物名称等方式查找相关学术资源。
scholar.google.com 收录
Canadian Census
**Overview** The data package provides demographics for Canadian population groups according to multiple location categories: Forward Sortation Areas (FSAs), Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs), Federal Electoral Districts (FEDs), Health Regions (HRs) and provinces. **Description** The data are available through the Canadian Census and the National Household Survey (NHS), separated or combined. The main demographic indicators provided for the population groups, stratified not only by location but also for the majority by demographical and socioeconomic characteristics, are population number, females and males, usual residents and private dwellings. The primary use of the data at the Health Region level is for health surveillance and population health research. Federal and provincial departments of health and human resources, social service agencies, and other types of government agencies use the information to monitor, plan, implement and evaluate programs to improve the health of Canadians and the efficiency of health services. Researchers from various fields use the information to conduct research to improve health. Non-profit health organizations and the media use the health region data to raise awareness about health, an issue of concern to all Canadians. The Census population counts for a particular geographic area representing the number of Canadians whose usual place of residence is in that area, regardless of where they happened to be on Census Day. Also included are any Canadians who were staying in that area on Census Day and who had no usual place of residence elsewhere in Canada, as well as those considered to be 'non-permanent residents'. National Household Survey (NHS) provides demographic data for various levels of geography, including provinces and territories, census metropolitan areas/census agglomerations, census divisions, census subdivisions, census tracts, federal electoral districts and health regions. In order to provide a comprehensive overview of an area, this product presents data from both the NHS and the Census. NHS data topics include immigration and ethnocultural diversity; aboriginal peoples; education and labor; mobility and migration; language of work; income and housing. 2011 Census data topics include population and dwelling counts; age and sex; families, households and marital status; structural type of dwelling and collectives; and language. The data are collected for private dwellings occupied by usual residents. A private dwelling is a dwelling in which a person or a group of persons permanently reside. Information for the National Household Survey does not include information for collective dwellings. Collective dwellings are dwellings used for commercial, institutional or communal purposes, such as a hotel, a hospital or a work camp. **Benefits** - Useful for canada public health stakeholders, for public health specialist or specialized public and other interested parties. for health surveillance and population health research. for monitoring, planning, implementation and evaluation of health-related programs. media agencies may use the health regions data to raise awareness about health, an issue of concern to all canadians. giving the addition of longitude and latitude in some of the datasets the data can be useful to transpose the values into geographical representations. the fields descriptions along with the dataset description are useful for the user to quickly understand the data and the dataset. **License Information** The use of John Snow Labs datasets is free for personal and research purposes. For commercial use please subscribe to the [Data Library](https://www.johnsnowlabs.com/marketplace/) on John Snow Labs website. The subscription will allow you to use all John Snow Labs datasets and data packages for commercial purposes. **Included Datasets** - [Canadian Population and Dwelling by FSA 2011](https://www.johnsnowlabs.com/marketplace/canadian-population-and-dwelling-by-fsa-2011) - This Canadian Census dataset covers data on population, total private dwellings and private dwellings occupied by usual residents by forward sortation area (FSA). It is enriched with the percentage of the population or dwellings versus the total amount as well as the geographical area, province, and latitude and longitude. The whole Canada's population is marked as 100, referring to 100% for the percentages. - [Detailed Canadian Population Statistics by CMAs and CAs 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-cmas-and-cas-2011) - This dataset covers the population statistics of Canada by Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs). It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by FED 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-fed-2011) - This dataset covers the population statistics of Canada from 2011 by Federal Electoral District of 2013 Representation Order. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Health Region 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-health-region-2011) - This dataset covers the population statistics of Canada by health region. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Province 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-province-2011) - This dataset covers the population statistics of Canada by provinces and territories. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. **Data Engineering Overview** **We deliver high-quality data** - Each dataset goes through 3 levels of quality review - 2 Manual reviews are done by domain experts - Then, an automated set of 60+ validations enforces every datum matches metadata & defined constraints - Data is normalized into one unified type system - All dates, unites, codes, currencies look the same - All null values are normalized to the same value - All dataset and field names are SQL and Hive compliant - Data and Metadata - Data is available in both CSV and Apache Parquet format, optimized for high read performance on distributed Hadoop, Spark & MPP clusters - Metadata is provided in the open Frictionless Data standard, and its every field is normalized & validated - Data Updates - Data updates support replace-on-update: outdated foreign keys are deprecated, not deleted **Our data is curated and enriched by domain experts** Each dataset is manually curated by our team of doctors, pharmacists, public health & medical billing experts: - Field names, descriptions, and normalized values are chosen by people who actually understand their meaning - Healthcare & life science experts add categories, search keywords, descriptions and more to each dataset - Both manual and automated data enrichment supported for clinical codes, providers, drugs, and geo-locations - The data is always kept up to date – even when the source requires manual effort to get updates - Support for data subscribers is provided directly by the domain experts who curated the data sets - Every data source’s license is manually verified to allow for royalty-free commercial use and redistribution. **Need Help?** If you have questions about our products, contact us at [info@johnsnowlabs.com](mailto:info@johnsnowlabs.com).
Databricks 收录
中国气象数据
本数据集包含了中国2023年1月至11月的气象数据，包括日照时间、降雨量、温度、风速等关键数据。通过这些数据，可以深入了解气象现象对不同地区的影响，并通过可视化工具揭示中国的气温分布、降水情况、风速趋势等。
github 收录
HazyDet
HazyDet是由解放军工程大学等机构创建的一个大规模数据集，专门用于雾霾场景下的无人机视角物体检测。该数据集包含383,000个真实世界实例，收集自自然雾霾环境和正常场景中人工添加的雾霾效果，以模拟恶劣天气条件。数据集的创建过程结合了深度估计和大气散射模型，确保了数据的真实性和多样性。HazyDet主要应用于无人机在恶劣天气条件下的物体检测，旨在提高无人机在复杂环境中的感知能力。
arXiv 收录
UniProt
UniProt（Universal Protein Resource）是全球公认的蛋白质序列与功能信息权威数据库，由欧洲生物信息学研究所（EBI）、瑞士生物信息学研究所（SIB）和美国蛋白质信息资源中心（PIR）联合运营。该数据库以其广度和深度兼备的蛋白质信息资源闻名，整合了实验验证的高质量数据与大规模预测的自动注释内容，涵盖从分子序列、结构到功能的全面信息。UniProt核心包括注释详尽的UniProtKB知识库（分为人工校验的Swiss-Prot和自动生成的TrEMBL），以及支持高效序列聚类分析的UniRef和全局蛋白质序列归档的UniParc。其卓越的数据质量和多样化的检索工具，为基础研究和药物研发提供了无可替代的支持，成为生物学研究中不可或缺的资源。
www.uniprot.org 收录