AssistQ

我们构建了AssistQ数据集，基于AI助手应该从教学视频和脚本中学习，以逐步指导用户。数据集包括从100新拍摄的第一人称视频导出的529问答样本。 每个问题都带有多步候选答案，可以通过从视觉细节 (例如，按钮的位置) 和纹理细节 (例如，按/转动作) 中推断出教学视频来完成。We constructed the AssistQ dataset, which targets the scenario where AI assistants learn from instructional videos and scripts to provide step-by-step guidance for users. The dataset comprises 529 question-answer pairs derived from 100 newly-shot first-person instructional videos. Each question is paired with multi-step candidate answers, which can be completed by inferring relevant information from the instructional videos based on visual cues (e.g., 'the position of buttons') and texture-related features (e.g., 'pressing or twisting motions').

搜集汇总

背景与挑战

背景概述

AssistQ数据集由新加坡国立大学于2022年发布，包含基于100个第一人称教学视频导出的529个问答样本。该数据集旨在通过视觉和纹理细节推断多步答案，以支持AI助手逐步指导用户。