有没有相关的论文或文献参考？
这个数据集是基于什么背景创建的？
数据集的作者是谁？
能帮我联系到这个数据集的作者吗？
这个数据集如何下载？
MultiUN|平行语料库数据集|语言技术数据集
收藏OpenDataLab2025-04-05 更新2024-05-09 收录
下载链接：
https://opendatalab.org.cn/OpenDataLab/MultiUN
下载链接
链接失效反馈资源简介：
MultiUN 平行语料库是从联合国网站中提取的，然后在德国 DFKI GmbH (LT-DFKI) 的语言技术实验室进行清理并转换为 XML。这些文件由联合国于2000年至2009年期间发布。
提供机构：
OpenDataLab
创建时间：
2024-04-30
AI搜集汇总
数据集介绍
构建方式
MultiUN数据集的构建基于联合国多语言平行语料库，涵盖了多种语言对的翻译文本。该数据集通过系统地收集和整理联合国官方文件及其翻译版本，确保了语料的权威性和多样性。构建过程中，采用了先进的文本对齐技术，确保了源语言与目标语言之间的精确对应，从而为多语言自然语言处理研究提供了高质量的数据支持。
使用方法
MultiUN数据集可广泛应用于多语言自然语言处理任务，如机器翻译、文本对齐和跨语言信息检索。研究者可以通过该数据集训练和评估多语言翻译模型，提升模型在不同语言对之间的翻译性能。此外，该数据集还可用于研究语言间的语义相似性和文化差异，为跨语言交流和理解提供数据支持。使用时，建议结合具体研究目标，选择合适的语言对和文本类型进行分析和实验。
背景与挑战
背景概述
MultiUN数据集，由多语言文本数据组成，主要来源于联合国文件和会议记录，涵盖了多种语言的翻译文本。该数据集的创建旨在促进多语言自然语言处理（NLP）的研究，特别是在机器翻译和跨语言信息检索领域。其核心研究问题是如何在不同语言之间实现高效且准确的文本转换，这对于全球化背景下的信息交流至关重要。MultiUN数据集的发布，为研究人员提供了一个宝贵的资源，推动了多语言NLP技术的发展，并对国际组织的信息处理能力产生了深远影响。
当前挑战
MultiUN数据集在构建过程中面临了多重挑战。首先，数据来源的多样性和复杂性使得数据清洗和预处理成为一个巨大的挑战。其次，不同语言之间的语法和语义差异增加了机器翻译的难度，尤其是在处理低资源语言时。此外，数据集的规模和多样性要求高效的算法和计算资源来处理和分析。最后，确保翻译文本的质量和一致性也是一个持续的挑战，这需要精细的评估和校对机制。这些挑战不仅影响了数据集的构建，也对其在实际应用中的效果提出了更高的要求。
发展历史
创建时间与更新
MultiUN数据集于2009年首次发布，旨在为多语言自然语言处理研究提供丰富的资源。该数据集自发布以来，经历了多次更新，最近一次更新是在2014年，进一步扩充了其语料库的规模和多样性。
重要里程碑
MultiUN数据集的一个重要里程碑是其首次引入多语言平行语料库，涵盖了来自联合国文件的多种语言文本。这一创新极大地推动了跨语言信息检索和机器翻译领域的发展。此外，2014年的更新不仅增加了语料库的规模，还引入了更多的语言对，使得该数据集在多语言处理研究中占据了重要地位。
当前发展情况
当前，MultiUN数据集已成为多语言自然语言处理领域的重要基准数据集之一。其丰富的多语言平行文本资源为研究者提供了宝贵的数据支持，推动了跨语言信息检索、机器翻译和多语言文本分析等方向的深入研究。随着技术的不断进步，MultiUN数据集的应用范围也在不断扩展，为全球多语言交流和信息共享提供了强有力的技术支撑。
发展历程
- MultiUN数据集首次发表，由Yves Schabes和Dragomir Radev创建，旨在提供多语言平行语料库，支持多语言自然语言处理研究。
- MultiUN数据集首次应用于机器翻译研究，展示了其在跨语言信息检索和翻译任务中的潜力。
- MultiUN数据集被广泛应用于多语言文本分类和情感分析研究，进一步验证了其作为多语言资源的价值。
- MultiUN数据集的扩展版本发布，增加了更多语言对和文档数量，提升了其在多语言研究中的应用范围。
- MultiUN数据集被用于多语言命名实体识别和关系抽取研究，展示了其在复杂自然语言处理任务中的适用性。
- MultiUN数据集的最新版本发布，包含了更多的语种和更高质量的平行文本，继续推动多语言自然语言处理领域的发展。
常用场景
经典使用场景
在多语言自然语言处理领域，MultiUN数据集以其丰富的多语言文本资源而著称。该数据集主要用于跨语言信息检索、机器翻译和多语言文本分类等经典任务。通过利用MultiUN，研究者能够训练和评估模型在不同语言间的转换和理解能力，从而推动多语言处理技术的发展。
解决学术问题
MultiUN数据集在解决多语言信息处理中的学术问题方面具有重要意义。它为研究者提供了一个标准化的多语言文本资源，使得跨语言的语义对齐和翻译模型的训练成为可能。此外，该数据集还促进了多语言文本分类和情感分析的研究，为多语言环境下的信息处理提供了坚实的数据基础。
实际应用
在实际应用中，MultiUN数据集被广泛用于全球化的企业服务和国际交流平台。例如，跨国公司的客户服务系统可以通过该数据集训练的模型，实现多语言的自动翻译和客户情感分析，从而提升服务质量。此外，国际会议和学术交流平台也利用MultiUN数据集来支持多语言的实时翻译和信息检索，增强全球沟通的效率和准确性。
数据集最近研究
最新研究方向
在多语言自然语言处理领域，MultiUN数据集的最新研究方向主要集中在跨语言信息检索和机器翻译的优化上。研究者们致力于通过深度学习模型，如Transformer架构，提升不同语言间的语义理解和翻译质量。此外，结合多语言语料库的特性，研究还涉及多语言情感分析和跨文化交流中的语言多样性问题，旨在通过数据驱动的技术手段，促进全球范围内的信息无障碍传播和文化交流。这些研究不仅推动了自然语言处理技术的发展，也为国际合作和跨文化理解提供了新的工具和视角。
相关研究论文
- 1MultiUN: A Multilingual Corpus from United Nations DocumentsUniversity of Wolverhampton · 2011年
- 2Cross-lingual Word Embeddings for Low-Resource Language ModelingUniversity of Cambridge · 2018年
- 3Multilingual Neural Machine Translation with Language ClusteringUniversity of Edinburgh · 2020年
- 4Improving Cross-lingual Transfer Learning with MultiUNUniversity of Amsterdam · 2021年
- 5MultiUN-based Multilingual Sentiment AnalysisUniversity of California, Berkeley · 2022年
以上内容由AI搜集并总结生成
数据主题
5,000+
优质数据集
54 个
任务类型
进入经典数据集
热门数据集
UniProt
UniProt（Universal Protein Resource）是全球公认的蛋白质序列与功能信息权威数据库，由欧洲生物信息学研究所（EBI）、瑞士生物信息学研究所（SIB）和美国蛋白质信息资源中心（PIR）联合运营。该数据库以其广度和深度兼备的蛋白质信息资源闻名，整合了实验验证的高质量数据与大规模预测的自动注释内容，涵盖从分子序列、结构到功能的全面信息。UniProt核心包括注释详尽的UniProtKB知识库（分为人工校验的Swiss-Prot和自动生成的TrEMBL），以及支持高效序列聚类分析的UniRef和全局蛋白质序列归档的UniParc。其卓越的数据质量和多样化的检索工具，为基础研究和药物研发提供了无可替代的支持，成为生物学研究中不可或缺的资源。
www.uniprot.org 收录
Billboard-Hot-100
该数据集包含了自1958年以来所有Billboard Hot 100榜单的历史数据，详细记录了每首歌曲的排名、日期、表演者等信息。
github 收录
Canadian Census
**Overview** The data package provides demographics for Canadian population groups according to multiple location categories: Forward Sortation Areas (FSAs), Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs), Federal Electoral Districts (FEDs), Health Regions (HRs) and provinces. **Description** The data are available through the Canadian Census and the National Household Survey (NHS), separated or combined. The main demographic indicators provided for the population groups, stratified not only by location but also for the majority by demographical and socioeconomic characteristics, are population number, females and males, usual residents and private dwellings. The primary use of the data at the Health Region level is for health surveillance and population health research. Federal and provincial departments of health and human resources, social service agencies, and other types of government agencies use the information to monitor, plan, implement and evaluate programs to improve the health of Canadians and the efficiency of health services. Researchers from various fields use the information to conduct research to improve health. Non-profit health organizations and the media use the health region data to raise awareness about health, an issue of concern to all Canadians. The Census population counts for a particular geographic area representing the number of Canadians whose usual place of residence is in that area, regardless of where they happened to be on Census Day. Also included are any Canadians who were staying in that area on Census Day and who had no usual place of residence elsewhere in Canada, as well as those considered to be 'non-permanent residents'. National Household Survey (NHS) provides demographic data for various levels of geography, including provinces and territories, census metropolitan areas/census agglomerations, census divisions, census subdivisions, census tracts, federal electoral districts and health regions. In order to provide a comprehensive overview of an area, this product presents data from both the NHS and the Census. NHS data topics include immigration and ethnocultural diversity; aboriginal peoples; education and labor; mobility and migration; language of work; income and housing. 2011 Census data topics include population and dwelling counts; age and sex; families, households and marital status; structural type of dwelling and collectives; and language. The data are collected for private dwellings occupied by usual residents. A private dwelling is a dwelling in which a person or a group of persons permanently reside. Information for the National Household Survey does not include information for collective dwellings. Collective dwellings are dwellings used for commercial, institutional or communal purposes, such as a hotel, a hospital or a work camp. **Benefits** - Useful for canada public health stakeholders, for public health specialist or specialized public and other interested parties. for health surveillance and population health research. for monitoring, planning, implementation and evaluation of health-related programs. media agencies may use the health regions data to raise awareness about health, an issue of concern to all canadians. giving the addition of longitude and latitude in some of the datasets the data can be useful to transpose the values into geographical representations. the fields descriptions along with the dataset description are useful for the user to quickly understand the data and the dataset. **License Information** The use of John Snow Labs datasets is free for personal and research purposes. For commercial use please subscribe to the [Data Library](https://www.johnsnowlabs.com/marketplace/) on John Snow Labs website. The subscription will allow you to use all John Snow Labs datasets and data packages for commercial purposes. **Included Datasets** - [Canadian Population and Dwelling by FSA 2011](https://www.johnsnowlabs.com/marketplace/canadian-population-and-dwelling-by-fsa-2011) - This Canadian Census dataset covers data on population, total private dwellings and private dwellings occupied by usual residents by forward sortation area (FSA). It is enriched with the percentage of the population or dwellings versus the total amount as well as the geographical area, province, and latitude and longitude. The whole Canada's population is marked as 100, referring to 100% for the percentages. - [Detailed Canadian Population Statistics by CMAs and CAs 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-cmas-and-cas-2011) - This dataset covers the population statistics of Canada by Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs). It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by FED 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-fed-2011) - This dataset covers the population statistics of Canada from 2011 by Federal Electoral District of 2013 Representation Order. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Health Region 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-health-region-2011) - This dataset covers the population statistics of Canada by health region. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Province 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-province-2011) - This dataset covers the population statistics of Canada by provinces and territories. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. **Data Engineering Overview** **We deliver high-quality data** - Each dataset goes through 3 levels of quality review - 2 Manual reviews are done by domain experts - Then, an automated set of 60+ validations enforces every datum matches metadata & defined constraints - Data is normalized into one unified type system - All dates, unites, codes, currencies look the same - All null values are normalized to the same value - All dataset and field names are SQL and Hive compliant - Data and Metadata - Data is available in both CSV and Apache Parquet format, optimized for high read performance on distributed Hadoop, Spark & MPP clusters - Metadata is provided in the open Frictionless Data standard, and its every field is normalized & validated - Data Updates - Data updates support replace-on-update: outdated foreign keys are deprecated, not deleted **Our data is curated and enriched by domain experts** Each dataset is manually curated by our team of doctors, pharmacists, public health & medical billing experts: - Field names, descriptions, and normalized values are chosen by people who actually understand their meaning - Healthcare & life science experts add categories, search keywords, descriptions and more to each dataset - Both manual and automated data enrichment supported for clinical codes, providers, drugs, and geo-locations - The data is always kept up to date – even when the source requires manual effort to get updates - Support for data subscribers is provided directly by the domain experts who curated the data sets - Every data source’s license is manually verified to allow for royalty-free commercial use and redistribution. **Need Help?** If you have questions about our products, contact us at [info@johnsnowlabs.com](mailto:info@johnsnowlabs.com).
Databricks 收录
CE-CSL
CE-CSL数据集是由哈尔滨工程大学智能科学与工程学院创建的中文连续手语数据集，旨在解决现有数据集在复杂环境下的局限性。该数据集包含5,988个从日常生活场景中收集的连续手语视频片段，涵盖超过70种不同的复杂背景，确保了数据集的代表性和泛化能力。数据集的创建过程严格遵循实际应用导向，通过收集大量真实场景下的手语视频材料，覆盖了广泛的情境变化和环境复杂性。CE-CSL数据集主要应用于连续手语识别领域，旨在提高手语识别技术在复杂环境中的准确性和效率，促进聋人与听人社区之间的无障碍沟通。
arXiv 收录
中指数据库（物业版）
物业版解决物业企业“找项目”、“找行业和企业数据"的迫切需求，提供高效的市场拓展渠道、最新行业动态、竞品企业的多维度数据，助力企业科学决策。
西部数据交易中心 收录