Youku-mPLUG

Youku-mPLUG预训练数据集挖掘自优酷站内海量的优质短视频内容，包含千万级别约36TB的视频、文本数据。其中视频均为覆盖10～120秒的UGC短视频内容，文本为视频对应的描述标题，长度5～30不等。该数据集抽取时品类均衡，内容共包含45个大类：电视剧剪辑、电视剧周边、电影剪辑、电影周边、综艺、相声小品、纪录片、传统文化、动漫、MV、翻唱、乐器演奏、健身、街舞、广场舞、竞技体育、篮球、足球、财经、科技、汽车、科学科普、生活百科、日常生活、搞笑、学历教育、游戏、职业职场、美食测评、美食制作、美容护肤、美妆、穿搭、旅游、宠物、家居装饰、房产装修、医疗健康、养生保健、三农、萌娃日常、亲子育儿、少儿才艺、少儿动漫、少儿玩具。

下游任务数据集
我们提供了3个不同的下游多模态视频Benchmark视频数据集去衡量预训练模型的能力。3个不同的任务具体包含：
类目预测：给定视频和对应视频标题，对该视频的类目进行预测。
视频检索：在给定一些视频以及一些文本的情况下，使用视频对文本进行检索和使用文本对视频进行检索。
视频描述：在给定视频的情况下，对视频中的画面进行描述。

The Youku-mPLUG pre-trained dataset is curated from massive high-quality short-form video content on Youku's official platform, comprising tens of millions of video and text samples with a total storage volume of approximately 36 TB. All included videos are user-generated content (UGC) short-form works with durations ranging from 10 to 120 seconds, while the accompanying text consists of the descriptive titles for each video, with lengths varying between 5 and 30 characters. The dataset is constructed with balanced category distribution, covering a total of 45 major categories: TV Drama Clips, TV Drama Peripherals, Movie Clips, Movie Peripherals, Variety Shows, Cross-talk and Skits, Documentaries, Traditional Culture, Animation, Music Videos (MVs), Cover Versions, Instrumental Performances, Fitness Training, Street Dance, Square Dance, Competitive Sports, Basketball, Football, Finance, Technology, Automobiles, Science Popularization, Life Encyclopedias, Daily Life, Humorous Content, Academic Education, Games, Professional Workplace, Food Reviews, Food Preparation, Skincare, Makeup, Clothing Styling, Travel, Pets, Home Decoration, Real Estate Decoration, Medical Health, Health Preservation, Agriculture, Rural Areas and Farmers, Daily Life of Cute Babies, Parenting & Childcare, Children's Talent Shows, Children's Animation, Children's Toys.

## Downstream Task Datasets
We provide three distinct multimodal video benchmark datasets to evaluate the performance of pre-trained models. The three specific tasks are detailed below:
1. **Category Prediction**: Given a video and its corresponding descriptive title, predict the category of the video.
2. **Video Retrieval**: Given a set of videos and text queries, perform cross-modal retrieval, including retrieving relevant texts using videos and retrieving relevant videos using texts.
3. **Video Captioning**: Given a video, generate a natural language descriptive caption for the visual content presented in the video.

搜集汇总

背景与挑战

背景概述

Youku-mPLUG是一个大规模的中文视频文本数据集，包含千万级别的视频和文本数据，涵盖45个类别，适用于多模态预训练和下游任务测试。数据集还提供了类目预测、视频检索和视频描述三个下游任务的数据集，用于评估模型性能。