openbmb/CAGUI

CAGUI是一个面向中文Android GUI的真实世界基准数据集，旨在评估GUI agent模型在两个互补的能力上的表现：Grounding和Agent。Grounding涉及理解GUI组件并将它们与语义关联起来，而Agent则是根据自然语言指令执行应用内的多步骤操作以实现用户目标。该数据集包括用于Grounding任务的截图和两个JSONL文件（cap.jsonl和ocr.jsonl），以及用于Agent任务的每集JSON文件和步骤级别的截图。

CAGUI is a real-world Chinese Android GUI benchmark designed to evaluate GUI agent models on two complementary capabilities: Grounding and Agent. Grounding involves understanding GUI components and associating them with semantics, while Agent refers to executing multi-step operations within the app based on natural language instructions to achieve user goals. The dataset includes screenshots and two JSONL files (cap.jsonl and ocr.jsonl) for the Grounding task, as well as episode JSON files and step-level screenshots for the Agent task.