遇见数据集

Krystalan/xmediasum

收藏
Hugging Face2023-02-15 更新2024-03-04 收录
官方服务:

资源简介:

--- annotations_creators: - expert-generated language: - en - zh - de language_creators: - crowdsourced license: - cc-by-nc-sa-4.0 multilinguality: - multilingual pretty_name: xmediasum size_categories: - 10K<n<100K source_datasets: - original tags: [] task_categories: - summarization task_ids: [] --- # Dataset Card for XMediaSum ### Dataset Summary We present XMediaSum, a cross-lingual dialogue summarization dataset with 40K English(dialogues)->Chinese(summaries) and 40K English (dialogues)->German(summaries) samples. XMediaSum is created by manually translating the English summaries of MediaSum (a English monolingual dialogue summarization dataset) to both Chinese and German. - Paper: [ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization](https://aclanthology.org/2022.emnlp-main.526/) (EMNLP 2022) - GitHub: https://github.com/krystalan/ClidSum ### Supported Task - Cross-Lingual Summarization - Cross-Lingual Dialogue Summarization ### Languages - source language: English - target language: Chinese and German ## Dataset Structure ### Data Instances One example is given below in JSON format: ```json { "dialogue": "MADELELEINE BRAND, host: OK, here's some good news on the jobs front for both men and women. A new survey out today from the employment firm Manpower finds that about a quarter of employers will add jobs this summer. That's for adults, but for teenagers this summer's job market is shaping up to be the weakest in more than 50 years.\r\nALEX COHEN, host: So, how do you get your teenage kids not to spend the entire summer glued to the couch? You're about to get some tips from Michelle Singletary. She's Day to Day's personal finance contributor. Hi, Michelle!\r\nMICHELLE SINGLETARY: Hi!\r\nALEX COHEN, host: So why is the summer job market so hard for teens this year?\r\nMICHELLE SINGLETARY: Lot of things going on right now. We've got a tough economy. We've got a lot of college graduates going into the market. We have people who are losing their jobs and taking jobs that would traditionally go to teens, like in restaurants and retailers. And we have a lot of older people holding on to their jobs and not retiring because they can't afford to retire. And that puts teens at the end of the line when it comes to these types of jobs.\r\nALEX COHEN, host: So you've got a teenager at home, a little bit young for the working world just yet, but what would you say to a teenager who's out there hunting around for a job?\r\nMICHELLE SINGLETARY: If you absolutely need a job, keep looking. You know, obviously the types of jobs that teens tend to go for in retail, fast food, you know, they still need people. And oftentimes you know, listen, you may not get the job at the beginning of the summer, but hold on because in late summer, when some of those college students are going back and perhaps some of those people who lost their jobs are finding permanent positions with more pay, you might be able to still get that job. So don't give up, you may spend a month or month and a half without it, but go back to those retailers and those restaurants and those fast food places to see if they still need someone.\r\nALEX COHEN, host: And now I know parents like having the break from providing allowance. But, you know, is - are there reasons maybe not to push your teen towards taking a job?\r\nMICHELLE SINGLETARY: I think it absolutely is. In fact I think too many teens are working and they don't need to work. They're some who absolutely need, they're contributing to their household or they're putting money into their own college fund. But more often than not, what parents do is say you've got to get a job, and then the teens get the job and they spend all the money on clothes and you know videos and iPods and paying their cell phone bills because they don't need a cell phone anyway.\r\nALEX COHEN, host: So it's not going towards the college tuition at all.\r\nMICHELLE SINGLETARY: It is not. It's just disposable income that they're disposing of. And parents are not setting any limits and you know and then the kids get used to the fact that they're using all of their paycheck. That's another bad habit. Because they don't have to pay bills and all, all their income goes through you know this stuff.\r\nMICHELLE SINGLETARY: And when it comes time to get a real job, they're surprised they don't have enough money. And so you know what? You can wait to work. Instead, maybe they can spend the summer volunteering at a charitable organization or you know going back to school and boosting up their math skills or their English skills. We push the teens out into the market too soon, I think for some families.\r\nALEX COHEN, host: But now let's say your kid is working. What tips can parents provide in terms of holding on to that summer money?\r\nMICHELLE SINGLETARY: You know, before they get their job, they need to sit down with them and do a budget. So before they actually work and get that first paycheck I mean, you know, have them draw up a budge where the money is going. And you ought to have some requirements for some of their money. That's right, be a parent.\r\nMICHELLE SINGLETARY: So make them put some of it towards their college fund, if in fact they're headed for college. You know what? Make them put some away, I call it the tax fund, even though they may not have to pay taxes, but to pay for long-term things that they may want. You know, books once they get to college, or maybe they want to get a car, and they can actually pay cash for it, with some of these funds. Don't let them just go out and spend it on movies and stuff. You ought to set some guidelines - this is where you should put the money. And look at their budget.\r\nALEX COHEN, host: Day to Day's personal finance contributor Michelle Singletary. Thank you, Michelle!\r\nMICHELLE SINGLETARY: You're welcome.\r\nALEX COHEN, host: Stay with us. NPR's Day to Day continues.", "summary": "The tight job market could be bad news for teens seeking summer work. If your teen does find a job, will he or she know how to manage those paychecks? Our personal finance contributor talks with Alex Cohen about ways to help teens find a job.", "summary_de": "Der angespannte Arbeitsmarkt könnte für Jugendliche, die Sommerarbeit suchen, eine schlechte Nachricht sein. Wenn Ihr Teenager einen Job findet, wird er oder sie wissen, wie er mit diesen Gehaltsschecks umgeht? Unser Mitarbeiter für persönliche Finanzen spricht mit Alex Cohen darüber, wie Teenager bei der Jobsuche unterstützt werden können.", "summary_zh": "紧张的就业市场对寻找暑期工作的青少年来说可能是个坏消息。如果你的孩子找到了一份工作,他/她懂得怎么管理这些薪水吗?我们的个人理财撰稿人与亚历克斯·科恩谈论如何帮助青少年找到工作。" }, ``` ### Data Fields - 'dialogue': An English dialogue - 'summary': the original English summary of the corresponding dialogue (provided by MediaSum) - 'summary_de': the human-translated German summary - 'summary_zh': the human-translated Chinese summary ### Data Splits - training set: 20K samples - validation set: 10K samples - testing set: 10K samples ## Dataset Creation Please refer to [our paper](https://aclanthology.org/2022.emnlp-main.526/) for more details. ## Considerations for Using the Data Please refer to [our paper](https://aclanthology.org/2022.emnlp-main.526/) for more details. ## Additional Information ### Dataset Curators [More Information Needed](https://github.com/krystalan/ClidSum) ### Licensing Information License: CC BY-NC-SA 4.0 ### Citation Information ``` @inproceedings{wang-etal-2022-clidsum, title = "{C}lid{S}um: A Benchmark Dataset for Cross-Lingual Dialogue Summarization", author = "Wang, Jiaan and Meng, Fandong and Lu, Ziyao and Zheng, Duo and Li, Zhixu and Qu, Jianfeng and Zhou, Jie", booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing", month = dec, year = "2022", address = "Abu Dhabi, United Arab Emirates", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.emnlp-main.526", pages = "7716--7729", abstract = "We present ClidSum, a benchmark dataset towards building cross-lingual summarization systems on dialogue documents. It consists of 67k+ dialogue documents and 112k+ annotated summaries in different target languages. Based on the proposed ClidSum, we introduce two benchmark settings for supervised and semi-supervised scenarios, respectively. We then build various baseline systems in different paradigms (pipeline and end-to-end) and conduct extensive experiments on ClidSum to provide deeper analyses. Furthermore, we propose mDialBART which extends mBART via further pre-training, where the multiple objectives help the pre-trained model capture the structural characteristics as well as key content in dialogues and the transformation from source to the target language. Experimental results show the superiority of mDialBART, as an end-to-end model, outperforms strong pipeline models on ClidSum. Finally, we discuss specific challenges that current approaches faced with this task and give multiple promising directions for future research. We have released the dataset and code at https://github.com/krystalan/ClidSum.", } ``` ### Contributions Thanks to [@krystalan](https://github.com/krystalan) for adding this dataset.

提供机构:
Krystalan
原始信息汇总

数据集概述

名称: XMediaSum

描述: XMediaSum是一个跨语言对话摘要数据集,包含40,000个英语对话到中文摘要的样本和40,000个英语对话到德语摘要的样本。该数据集通过人工将MediaSum(一个英语单语对话摘要数据集)的英语摘要翻译成中文和德语创建。

语言:

  • 源语言: 英语
  • 目标语言: 中文和德语

任务:

  • 跨语言摘要
  • 跨语言对话摘要

数据集结构

数据实例: 每个实例包含以下字段:

  • dialogue: 英语对话
  • summary: 原始英语摘要
  • summary_de: 人工翻译的德语摘要
  • summary_zh: 人工翻译的中文摘要

数据分割:

  • 训练集: 20,000样本
  • 验证集: 10,000样本
  • 测试集: 10,000样本

许可证

许可证: CC BY-NC-SA 4.0

搜集汇总
数据集介绍
Krystalan/xmediasum 数据集图片
构建方式
跨语言对话摘要任务在自然语言处理领域具有重要研究价值,然而高质量标注数据的匮乏长期制约着该领域的发展。XMediaSum数据集正是为填补这一空白而构建,其以英文单语对话摘要数据集MediaSum为基础,通过人工翻译方式将原始英文摘要精准转化为中文和德文版本。具体而言,研究团队从MediaSum中选取了40,000条英文对话样本,并分别聘请母语为中文和德语的翻译专家,在严格的质量控制流程下完成摘要的翻译工作,最终形成包含80,000条跨语言对话摘要样本的高质量数据集。
使用方法
XMediaSum数据集适用于多种跨语言对话摘要研究场景。研究者可直接加载数据集中的'dialogue'字段作为源输入,配合'summary_zh'或'summary_de'作为目标输出,构建端到端的跨语言摘要模型。数据集支持监督学习与半监督学习两种范式,其中监督学习场景可利用全部标注数据训练模型,而半监督学习场景则允许研究者探索利用部分标注数据结合大规模无标注数据的训练策略。此外,mDialBART等预训练模型可基于该数据集进行进一步预训练,通过多任务学习目标捕捉对话结构特征与跨语言转换规律,从而提升摘要生成质量。
背景与挑战
背景概述
XMediaSum数据集由Jiaan Wang等研究人员于2022年在EMNLP会议上提出,其核心研究问题聚焦于跨语言对话摘要这一前沿领域。该数据集以MediaSum为基础,通过人工翻译将英文对话摘要精准转化为中文和德文,构建了包含40,000对英文对话与中文摘要、40,000对英文对话与德文摘要的高质量资源。这一开创性工作填补了跨语言对话摘要领域的数据空白,为多语言自然语言处理研究提供了重要基准,推动了对话系统与机器翻译技术的交叉融合,在学术和工业界产生了广泛影响。
当前挑战
XMediaSum数据集面临的挑战主要体现在两个方面。在领域问题层面,跨语言对话摘要需同时应对对话结构复杂性(如多轮交互、口语化表达)与跨语言语义一致性难题,现有模型在保持对话关键信息的同时,难以在目标语言中生成流畅且忠于原意的摘要。在构建过程中,人工翻译的准确性保障是核心难点,需克服语言歧义、文化差异及摘要长度控制等问题,同时确保40,000对样本的翻译质量与标注一致性,这对资源投入和流程管理提出了极高要求。
常用场景
经典使用场景
XMediaSum作为跨语言对话摘要领域的开创性数据集,其经典使用场景聚焦于构建和评估能够将英语对话自动转化为中文或德文摘要的模型。研究者可基于该数据集训练端到端的跨语言摘要系统,例如利用序列到序列框架,使模型在理解英语对话语义的同时,直接生成目标语言(中文或德文)的凝练摘要。该数据集还为半监督学习场景提供了基准,允许探索如何利用源语言摘要辅助目标语言生成,从而推动跨语言自然语言处理技术在多语种对话理解与生成中的深度融合。
解决学术问题
XMediaSum有效解决了跨语言对话摘要研究中高质量标注数据匮乏的学术难题。此前,对话摘要研究多局限于单语场景,而跨语言任务面临对话结构复杂与语言转换双重挑战。该数据集通过人工翻译MediaSum的英文摘要,构建了40K英文对话至中文摘要和40K英文对话至德文摘要的平行语料,为评估模型在跨语言场景下的语义保持能力、语言忠实度及摘要简洁性提供了标准化测试平台。其发布推动了跨语言摘要从理论探索走向可复现的实证研究,显著降低了领域入门门槛。
实际应用
在实际应用中,XMediaSum所支撑的跨语言对话摘要技术可赋能多语种会议记录自动总结、跨国客服对话简报生成及全球媒体访谈内容本地化等场景。例如,国际企业可利用该技术将英语商务会议录音摘要实时转化为中文或德文要点,提升跨团队协作效率;新闻机构可自动将英语播客摘要翻译为多语言版本,加速内容全球化分发。该数据集还助力开发多语种智能助手,使其能够从混合语言对话中提取关键信息并以用户母语呈现,优化跨文化信息交互体验。
数据集最近研究
最新研究方向
跨语言对话摘要正成为自然语言处理领域的前沿热点,XMediaSum数据集以其独特的双语摘要标注(英语对话至中文和德语摘要)为多语言对话理解提供了关键资源。该数据集源自MediaSum,通过人工翻译构建了40K对英中与英德样本,支撑了ClidSum等基准的建立。当前研究聚焦于端到端跨语言摘要模型,如mDialBART通过预训练目标优化对话结构捕捉与语言转换能力,在监督与半监督场景下显著超越流水线方法。这一方向与全球多语言信息处理需求紧密相关,尤其在跨国新闻对话、客服系统等场景中,推动着低资源语言摘要技术的突破。XMediaSum的发布不仅填补了跨语言对话摘要标注的空白,也为多语言语义对齐、零样本迁移等前沿探索提供了标准化评估平台,对促进包容性人工智能发展具有深远意义。
以上内容由遇见数据集搜集并总结生成
二维码
社区交流群
二维码
科研交流群
商业服务