MultiSubs (MultiSubs: A Large-scale Multimodal and Multilingual Dataset)

MultiSubs 是从 OPUS OpenSubtitles 数据集收集的多语言字幕数据集，该数据集又来自 opensubtitles.org。我们在字幕中用网络图像补充了一些文本片段（此版本中的视觉显着名词），其中片段的词义已使用跨语言方法消除歧义。我们引入了填空任务和词汇翻译任务来演示数据集的实用性。有关数据集和任务的更详细描述，请参阅我们的论文。 Multisubs 将有利于单词视觉基础的研究，尤其是在自由形式句子的上下文中。

MultiSubs is a multilingual subtitle dataset collected from the OPUS OpenSubtitles corpus, which itself originates from opensubtitles.org. We supplemented some text segments (visually salient nouns in this version) from the subtitles with web images, where the meanings of these segments have been disambiguated using cross-lingual methods. We introduce fill-in-the-blank tasks and lexical translation tasks to demonstrate the practical utility of this dataset. For a more detailed description of the dataset and the tasks, please refer to our associated paper. MultiSubs will facilitate research on visual grounding of words, particularly in the context of free-form sentences.

搜集汇总

背景与挑战

背景概述

MultiSubs是一个大规模多模态多语言数据集，基于OPUS OpenSubtitles字幕数据构建，通过补充网络图像来增强文本片段，用于单词视觉基础研究。该数据集由伦敦帝国理工学院和马托格罗索联邦大学于2021年发布，支持填空和词汇翻译等任务，遵循CC BY 4.0许可协议。