How2R Dataset

Amazon Mechanical Turk (AMT) is used to collect annotations on HowTo100M videos. 30k 60-second clips are randomly sampled from 9,421 videos and present each clip to the turkers, who are asked to select a video segment containing a single, self-contained scene. After this segment selection step, another group of workers are asked to write descriptions for each displayed segment. Narrations are not provided to the workers to ensure that their written queries are based on visual content only. These final video segments are 10-20 seconds long on average, and the length of queries ranges from 8 to 20 words. From this process, 51,390 queries are collected for 24k 60-second clips from 9,371 videos in HowTo100M, on average 2-3 queries per clip. The video clips and its associated queries are split into 80% train, 10% val and 10% test.

亚马逊机械式劳工（AMT）被用于收集对HowTo100M视频的标注。从9,421个视频中随机抽取了30,000个60秒的剪辑，并将每个剪辑呈现给众包工作者，要求他们选择包含单一、独立场景的视频片段。在此片段选择步骤之后，另一组工作者被要求为每个展示的片段撰写描述。为了保证工作者所撰写的查询仅基于视觉内容，并未提供叙述。这些最终的视频片段平均长度为10-20秒，查询的长度介于8至20词之间。通过此过程，从HowTo100M中的9,371个视频的24,000个60秒剪辑中收集了51,390个查询，平均每个剪辑包含2至3个查询。视频剪辑及其相关查询被划分为80%的训练集、10%的验证集和10%的测试集。