How2R

Amazon Mechanical Turk (AMT) 用于收集 HowTo100M 视频的注释。从 9,421 个视频中随机抽取 30,000 个 60 秒的剪辑，并将每个剪辑呈现给土耳其人，他们被要求选择一个包含单个独立场景的视频片段。在此段选择步骤之后，要求另一组工人为每个显示的段编写描述。不会向工作人员提供旁白，以确保他们的书面查询仅基于视觉内容。这些最终的视频片段平均长度为 10-20 秒，查询的长度范围为 8 到 20 个单词。在这个过程中，从 HowTo100M 中的 9,371 个视频中收集了 24k 60 秒剪辑的 51,390 个查询，平均每个剪辑有 2-3 个查询。视频剪辑及其相关查询分为 80% 训练、10% 验证和 10% 测试。Amazon Mechanical Turk (AMT) was employed to collect annotations for the HowTo100M video dataset. Specifically, 30,000 60-second clips were randomly sampled from 9,421 source videos, and each clip was presented to Turkers, who were instructed to select a single self-contained video segment. Following this segment selection step, another cohort of workers was asked to write descriptions for each displayed segment. No narration was provided to the workers, ensuring that their written queries were solely grounded in the visual content of the clips. These final selected video segments have an average duration of 10–20 seconds, with the query lengths ranging from 8 to 20 words. Throughout this annotation pipeline, 51,390 queries for 24,000 60-second clips were collected from 9,371 videos in the HowTo100M dataset, yielding an average of 2–3 queries per clip. The video clips and their corresponding queries were split into three subsets: 80% for training, 10% for validation, and 10% for testing.

搜集汇总

背景与挑战

背景概述

How2R数据集基于HowTo100M视频，通过Amazon Mechanical Turk收集了24,000个60秒剪辑的51,390个查询，平均每个剪辑有2-3个查询，数据划分为80%训练、10%验证和10%测试。该数据集用于视觉问答和视频字幕生成等任务，由微软于2020年发布。