TravelPlanner

# TravelPlanner Dataset

TravelPlanner is a benchmark crafted for evaluating language agents in tool-use and complex planning within multiple constraints. (See our [paper](https://arxiv.org/pdf/2402.01622.pdf) for more details.)

## Introduction

In TravelPlanner, for a given query, language agents are expected to formulate a comprehensive plan that includes transportation, daily meals, attractions, and accommodation for each day.

TravelPlanner comprises 1,225 queries in total. The number of days and hard constraints are designed to test agents' abilities across both the breadth and depth of complex planning.

## Split

<b>Train Set</b>: 5 queries with corresponding human-annotated plans for group, resulting in a total of 45 query-plan pairs. This set provides the human annotated plans as demonstrations for in-context learning.

<b>Validation Set</b>: 20 queries from each group, amounting to 180 queries in total. There is no human annotated plan in this set.

<b>Test Set</b>: 1,000 randomly distributed queries. To avoid data contamination, we only provide the level, days, and natural language query fields.

## Record Layout

- "org": The city from where the journey begins.
- "dest": The destination city.
- "days": The number of days planned for the trip.
- "visiting_city_number": The total number of cities included in the itinerary.
- "date": The specific date when the travel is scheduled.
- "people_number": The total number of people involved in the travel.
- "local_constraint": The local hard constraint, including house rule, cuisine, room type and transportation.
- "query": A natural language description or request related to the travel plan.
- "level": The difficulty level, which is determined by the number of hard constraints.
- "annotated_plan": A detailed travel plan annotated by a human, ensuring compliance with all common sense requirements and specific hard constraints.
- "reference_information": Reference information for "sole-planning" mode.

## Citation

If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries.

```bib
@article{Xie2024TravelPlanner,
author = {Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, Yu Su},
title = {TravelPlanner: A Benchmark for Real-World Planning with Language Agents},
journal = {arXiv preprint arXiv: 2402.01622},
year = {2024}
}
```

# TravelPlanner 数据集

TravelPlanner是专为评估语言智能体（AI Agent）在多约束条件下的工具使用与复杂规划能力而构建的基准测试集（更多细节参见我们的[论文](https://arxiv.org/pdf/2402.01622.pdf)）。

## 简介

在TravelPlanner数据集中，针对给定的查询请求，语言智能体需要制定一份涵盖交通、每日餐食、景点及每日住宿的综合旅行规划。该数据集总计包含1225条查询请求，其设计的行程天数与硬性约束条件，旨在从广度与深度两个维度检验智能体的复杂规划能力。

## 数据集划分

<b>训练集</b>：每组包含5条带有对应人类标注规划的查询，总计生成45条查询-规划对。该集合提供人类标注的规划作为上下文学习的演示示例。
<b>验证集</b>：每组抽取20条查询，总计180条查询。该集合未提供人类标注的规划方案。
<b>测试集</b>：1000条随机分布的查询。为避免数据污染，仅提供难度等级、行程天数与自然语言查询字段。

## 数据记录格式

- "org"：旅程起始城市
- "dest"：目的地城市
- "days"：行程规划的总天数
- "visiting_city_number"：行程中涵盖的城市总数
- "date"：旅行计划的具体日期
- "people_number"：参与旅行的总人数
- "local_constraint"：本地硬性约束，包括民宿规则、餐饮偏好、房型与交通方式要求
- "query"：与旅行规划相关的自然语言描述或请求
- "level"：难度等级，由硬性约束的数量决定
- "annotated_plan"：由人类标注的详细旅行规划，确保符合所有常识要求与特定硬性约束
- "reference_information"：“单一规划”模式的参考信息

## 引用说明

若本论文及相关资源对你的研究有所助益，请务必引用。如有任何疑问，欢迎随时与我们联系。

bib
@article{Xie2024TravelPlanner,
author = {Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, Yu Su},
title = {TravelPlanner: A Benchmark for Real-World Planning with Language Agents},
journal = {arXiv preprint arXiv: 2402.01622},
year = {2024}
}