orlando23/failed_agent_trajectory

该数据集用于验证LoT代理如何帮助识别移动执行轨迹中的错误操作。数据集包含52个具有特定任务目标的移动代理轨迹，这些轨迹来自MagicWand平台上的指令引导执行。每个文件夹包含一个执行轨迹，其中包括轨迹中的截图和描述每个执行步骤细节的JSON文件。

This dataset is prepared for verifying how LoT agent can help us identify incorrect actions taken in mobile execution trajectory. We select 52 mobile agent trajectories with specific task goals from instruction-guided executions on MagicWand platform. Each folder has an execution trajectory that includes screenshots along this trajectory, and json files describing the details of each execution step.

原始信息汇总

Mobile Trajectory Verification Data

数据集概述

• 用途: 用于验证LoT代理如何帮助识别移动执行轨迹中的错误操作。
• 来源: 从MagicWand平台上的指令引导执行中选择的52个移动代理轨迹。
• 内容: 每个文件夹包含一个执行轨迹，包括轨迹中的截图和描述每个执行步骤细节的json文件。

引用信息

• BibTeX: bibtex @misc{fan2024readpointedlayoutawaregui, title={Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding}, author={Yue Fan and Lei Ding and Ching-Chen Kuo and Shan Jiang and Yang Zhao and Xinze Guan and Jie Yang and Yi Zhang and Xin Eric Wang}, year={2024}, eprint={2406.19263}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2406.19263}, }

  @inproceedings{dingenhancing, title={Enhancing Mobile" How-to" Queries with Automated Search Results Verification and Reranking}, author={Ding, Lei and Zhang, Yi and Bheemanpally, Jeshwanth}, booktitle={The Second Workshop on Generative Information Retrieval} }