VideoGUI

VideoGUI是一个多模态基准数据集，专注于评估GUI助手的视觉中心任务能力。该数据集由新加坡国立大学Show Lab和微软Gen AI创建，包含从高质量网络教学视频中提取的11种视觉中心软件应用的86个复杂任务和463个子任务。数据集的创建过程涉及手动选择教学视频，参与者复制视频中的操作，并进行详细的手动标注。VideoGUI的应用领域主要集中在媒体创建、编辑和AI工具，旨在通过视觉中心的任务解决复杂的软件操作问题。

VideoGUI is a multimodal benchmark dataset dedicated to evaluating the visual-centric task capabilities of GUI assistants. Developed jointly by the Show Lab at the National University of Singapore and Microsoft Gen AI, the dataset is curated from high-quality web-based tutorial videos, encompassing 86 complex tasks and 463 subtasks spanning 11 visually focused software applications. The construction of VideoGUI involves three core procedures: manually selecting tutorial videos, having participants reproduce the operational steps demonstrated in the videos, and carrying out detailed manual annotations. Targeted at the domains of media creation, editing and AI tools, VideoGUI aims to resolve complex software operation problems through visual-centric task solving.