PARADISE

PARADISE数据集由约翰斯·霍普金斯大学和科奇大学创建，包含超过104,000条警告和提示，源自wikiHow的实用程序文本。该数据集旨在评估大型语言模型在推理和执行计划方面的能力，特别是在缺乏明确指令的情况下推断隐含知识。数据集涵盖广泛领域，通过问答格式进行任务设计，以测试模型对隐含计划知识的理解。PARADISE不仅为模型评估提供了一个可靠的测试平台，还促进了零样本和少样本学习在未见程序任务上的性能提升。

The PARADISE dataset was created by Johns Hopkins University and Koç University, comprising over 104,000 warnings and tips sourced from how-to utility texts on wikiHow. This dataset is designed to evaluate the capabilities of large language models (LLMs) in reasoning and plan execution, particularly in inferring implicit knowledge when explicit instructions are absent. Covering a wide range of domains, the dataset adopts a question-and-answer format for task design to test models' understanding of implicit planning knowledge. PARADISE not only provides a reliable testbed for model evaluation but also facilitates performance improvements of zero-shot and few-shot learning on unseen procedural tasks.

搜集汇总

背景与挑战

背景概述

PARADISE数据集由约翰斯·霍普金斯大学和科奇大学创建，包含超过10.4万条源自wikiHow的警告和提示，旨在评估大型语言模型在缺乏明确指令时推断隐含知识的能力。该数据集通过问答格式覆盖广泛领域，为模型测试和零样本、少样本学习性能提升提供了可靠平台。