BatsResearch/planetarium

Planetarium🪐数据集是一个用于评估大型语言模型（LLMs）在将自然语言描述的计划问题翻译成PDDL（Planning Domain Definition Language）方面的能力的数据集和基准。该数据集包含来自Blocks World和Gripper领域的PDDL和自然语言描述的计划问题对。任务是将各种初始和目标设置的自然语言描述翻译成正式的PDDL问题描述。数据集还描述了Blocks World和Gripper领域中的不同设置，如`on_table`、`stack`、`staircase`、`swap`、`tower`、`holding_one`、`equal_towers`、`invert`等。数据集的创建目的是为了展示LLMs在使用经典规划器解决计划问题时，虽然可以生成看似有效的PDDL问题描述，但这些描述往往并不正确。Planetarium🪐数据集通过图同构的方法，能够正确且稳健地测量从自然语言问题翻译而来的PDDL问题的正确性。Planetarium🪐 dataset is a dataset and benchmark designed to evaluate the capability of large language models (LLMs) to translate natural language-described planning problems into Planning Domain Definition Language (PDDL). This dataset contains paired planning problem descriptions in both PDDL and natural language from the Blocks World and Gripper domains. The core task is to translate natural language descriptions of various initial and goal configurations into formal PDDL problem specifications. The dataset also covers diverse configurations within the Blocks World and Gripper domains, such as `on_table`, `stack`, `staircase`, `swap`, `tower`, `holding_one`, `equal_towers`, `invert`, and others. The dataset was developed to demonstrate that while large language models can generate seemingly valid PDDL problem descriptions when solving planning tasks with classical planners, these generated descriptions are often incorrect. The Planetarium🪐 dataset can correctly and robustly measure the correctness of PDDL problems translated from natural language planning queries via graph isomorphism-based methods.

原始信息汇总

数据集概述

许可证

• 本数据集遵循CC BY 4.0许可证。

数据集信息

• 特征列表：
  • id: 数据类型为int64
  • name: 数据类型为string
  • domain: 数据类型为string
  • init: 数据类型为string
  • goal: 数据类型为string
  • num_objects: 数据类型为int64
  • problem_pddl: 数据类型为string
  • natural_language: 数据类型为string
  • init_is_abstract: 数据类型为int64
  • init_num_propositions: 数据类型为int64
  • goal_is_abstract: 数据类型为int64
  • goal_num_propositions: 数据类型为int64
  • is_placeholder: 数据类型为int64
  • split: 数据类型为int64

数据分割

• 训练集：
  • 文件大小: 247635764字节
  • 样本数量: 132037个

数据集大小

• 下载大小: 46923843字节
• 数据集大小: 247635764字节

配置

• 默认配置：
  • 数据文件路径: data/train-*