MoTIF

我们引入了具有迭代反馈 (MoTIF) 的新数据集移动应用任务，其目标是在移动应用中完成自然语言命令。移动应用程序提供了一个可扩展的领域，以研究VLN方法的实际下游使用。此外，移动应用程序命令提供交互式导航的指令，因为它们通过点击，键入或滑动导致状态变化的动作序列。MoTIF是第一个包含可行性注释的，其中包含二进制可行性标签和细粒度标签，以说明为什么任务无法满足。We present MoTIF (Mobile app Tasks with Iterative Feedback), a novel dataset focused on mobile application tasks where the goal is to execute natural language commands within mobile apps. Mobile applications constitute a scalable domain for investigating the practical downstream applications of VLN methods. Additionally, mobile app commands serve as instructions for interactive navigation, as they correspond to sequences of actions that alter the app's state through clicking, typing, or swiping. MoTIF is the first dataset to include feasibility annotations, which comprise both binary feasibility labels and fine-grained labels that explain why a task cannot be successfully completed.

搜集汇总

背景与挑战

背景概述

MoTIF是一个专注于移动应用任务的数据集，旨在通过自然语言命令完成交互式导航，并包含可行性注释。该数据集由伊利诺伊大学厄巴纳香槟分校等机构于2022年发布，相关资源可在GitHub和arXiv上获取。