KWAI-AD-AudVis

It consists of 85,432 ads videos from the China popular short-term video app, Kwai. The videos were made and uploaded by commercial advertisers rather than personal users. The reason to use the ads videos lied on two folds: 1) the source guarantees the videos under control to some level, such as high-resolution pictures and intention-ally designed scene; 2) the ads videos mimic the style of the ones uploaded by personal users, as they are played in be-tween the personal videos in Kwai app. It can be seen as a quality controlled UGVs dataset.The dataset was collected in two batches (Batch-1 is our preliminary work), coming with the tags of ads industry cluster. The videos were randomly picked from a pool. The pool was formed by selecting the ads from several contiguous days.Half of the selected ads had click through rate(CTR) in top30000 within that day and the other half had CTR in bottom30000. It should be noticed that the released dataset is a sub-set of the pool. The audio track had2 channels (we mixed to mono channel in the study) and was sampled at 44.1 kHz, while the visual track had resolution of1280×720 and was sampled at 25frame per second(FPS).This dataset is a extension of the KWAI-AD corpus [3]. It is not only suitable for tasks in multimodal learning area, but also for ones in ads recommendation. It shows that the ads videos have three main characteristics: 1) The videos may have very inconsistent information in visual or audio streams. For example, the video may play a drama-like story at first, and then present the product introduction, whose scenes are very different. 2) The correspondence between audio and visual streams is not clear.For instance, similar visual objects (e.g. talking salesman)come with very different audio streams. 3) The relationship between audio and video varies in different industries. For example, game or E-commerce ads will have very different styles. These characteristics make the dataset suitable yet challenging for our study about the AVC learning. In the folder, you will see: audio_features.tar.gz, meta, README, samples, ad_label.npy, video_fetaures.tar.gz. The details are included in README. If you use our dataset, please cite our paper: "Themes Inferred Audio-visual Correspondence Learning" (https://arxiv.org/pdf/2009.06573.pdf)

本数据集包含来自中国热门短视频应用快手（Kwai）的85432条广告视频。这些视频由商业广告主而非个人用户制作并上传。选用广告视频的原因有两点：其一，该来源可在一定程度上保证视频可控性，例如具备高分辨率画质与精心设计的场景；其二，此类广告视频的风格与个人用户上传的视频高度相似，因为它们会在快手应用的个人视频流之间插播。本数据集可视为经过质量管控的用户生成视频（User Generated Videos, UGVs）数据集。
数据集分为两批次采集（批次1为前期工作成果），附带广告行业集群标签。视频从候选池中随机选取，该候选池由连续多日的广告视频筛选而来。其中一半入选广告在当日的点击率（Click Through Rate, CTR）排名前30000，另一半则排名后30000。需要说明的是，公开发布的数据集为候选池的一个子集。
该数据集的音频轨道为双声道（本研究中将其混合为单声道），采样率为44.1kHz；视频轨道分辨率为1280×720，帧率为25帧每秒（Frames Per Second, FPS）。
本数据集是KWAI-AD语料库[3]的扩展版本，不仅适用于多模态学习领域的相关任务，也可用于广告推荐相关研究。该广告视频数据集具备三大核心特征：1）视频的视觉与音频流信息可能存在高度不一致，例如视频可能先播放类似剧集的故事，随后再展示产品介绍，二者场景差异显著；2）音频与视觉流之间的对应关系并不明确，例如相似的视觉画面（如正在讲解的销售人员）可能搭配截然不同的音频流；3）不同行业的音视频关联关系存在差异，例如游戏类与电商类广告的风格差异明显。这些特征使得本数据集既适用于我们针对音视频对应学习（Audio-Visual Correspondence Learning, AVC learning）的研究，同时也颇具挑战性。
数据集文件夹中包含以下文件：audio_features.tar.gz、meta、README、samples、ad_label.npy、video_fetaures.tar.gz，详细说明请参阅README文档。若使用本数据集，请引用以下论文："Themes Inferred Audio-visual Correspondence Learning"（https://arxiv.org/pdf/2009.06573.pdf）

搜集汇总

背景与挑战

背景概述

KWAI-AD-AudVis是一个包含85,432个Kwai广告视频的多模态数据集，具有高分辨率视频(1280×720)和高质量音频(44.1 kHz)特性。该数据集专为多模态学习和广告推荐研究设计，其独特的音频-视觉不一致性和行业差异特性使其成为具有挑战性的研究资源。