WikiHow任务集
收藏arXiv2024-02-24 更新2024-08-06 收录
下载链接：
http://arxiv.org/abs/2305.08144v3
下载链接
链接失效反馈官方服务：
资源简介：
WikiHow任务集是基于WikiHow应用构建的，旨在评估大型语言模型（LLMs）在图形用户界面（GUI）交互中的能力。该数据集包含150个多阶段任务，涵盖了搜索、浏览、理解和管理等多种GUI操作能力。数据集通过爬取WikiHow应用中的107,448个页面和856,045个文本及多媒体资源创建，确保了评估的一致性。创建过程中，使用了任务定义模板和ChatGPT辅助重写指令，以增加表达的多样性。WikiHow任务集的应用领域主要集中在评估LLMs在复杂环境中的多轮交互能力，解决实际GUI交互中的挑战。
The WikiHow Task Set is built on the WikiHow application, designed to evaluate the capabilities of Large Language Models (LLMs) in graphical user interface (GUI) interactions. The dataset contains 150 multi-stage tasks, covering a variety of GUI operation capabilities including searching, browsing, comprehension and management. It is constructed by crawling 107,448 pages and 856,045 text and multimedia resources from the WikiHow application, which ensures the consistency of the evaluation. During the creation process, task definition templates and ChatGPT were used to assist in rewriting instructions to increase the diversity of expressions. The application scenarios of the WikiHow Task Set mainly focus on evaluating the multi-turn interaction capabilities of LLMs in complex environments and addressing challenges in practical GUI interactions.
提供机构：
上海交通大学人工智能研究院
创建时间：
2023-05-14
