Multimodal C4 (mmc4)

Multimodal C4 (mmc4)是由University of California, Santa Barbara等机构开发的开放式十亿规模图像与文本交错数据集。该数据集通过线性分配算法，利用CLIP特征将图像嵌入到c4文本数据集中，覆盖烹饪、旅行、技术等多个日常主题。经过过滤不适当内容后，数据集包含1.012亿文档，其中交错嵌入了5.71亿图像和430亿英语词汇。mmc4旨在支持复杂的视觉语言模型训练，解决图像与文本交互的多样化问题，适用于少样本学习及多模态语言技术的发展。Multimodal C4 (mmc4) is an open, billion-scale interleaved image-text dataset developed by institutions including the University of California, Santa Barbara. This dataset embeds images into the C4 text corpus using CLIP features via a linear assignment algorithm, covering a range of daily topics such as cooking, travel, and technology. After filtering out inappropriate content, the dataset contains 101.2 million documents, which are interleaved with 571 million images and 43 billion English words. mmc4 is designed to support the training of complex vision-language models, address the diversity challenges of image-text interaction, and facilitate few-shot learning as well as the advancement of multimodal language technologies.

搜集汇总

背景与挑战

背景概述

Multimodal C4 (mmc4)是一个开放的多模态数据集，包含超过5.71亿张图像和430亿个标记，图像与文本交错排列。数据集提供不同版本，包括减少面部图像的版本，文档以jsonl格式存储，包含文本、图像URL及匹配信息，使用受ODC-BY和Common Crawl条款限制。