AitW Dataset

Android in the Wild (AitW) is a dataset for device-control research which is orders of magnitude larger than current datasets. The dataset contains human demonstrations of device interactions, including the screens and actions, and corresponding natural language instructions. It consists of 715k episodes spanning 30k unique instructions, four versions of Android (v10–13), and eight device types (Pixel 2 XL to Pixel 6) with varying screen resolutions. It contains multi-step tasks that require semantic understanding of language and visual context.

Android in the Wild (AitW) 数据集是一款针对设备控制研究的巨型数据集，其规模远超现有数据集。该数据集收录了人类对设备交互的演示，包括屏幕界面和操作行为，以及相应的自然语言指令。数据集包含 715,000 个场景，涵盖 30,000 个独特的指令，四个版本的 Android 操作系统（v10–v13），以及八种设备类型（从 Pixel 2 XL 到 Pixel 6），这些设备的屏幕分辨率各不相同。其中包含多步骤任务，这些任务需要语言语义理解和视觉上下文的理解。