ChinaTravel

# ChinaTravel Dataset

ChinaTravel is a benchmark meticulously designed to provide a comprehensive and scalable evaluation framework for language agents in multi-day multi-POI travel planning. See our [paper](https://arxiv.org/pdf/2412.13682) for more details.

## Introduction

In ChinaTravel, for a given query, language agents are expected to use the provided tools in sandbox to collect information and generate a travel plan in json format. The plan should include a list of POIs (restaurants, attractions, accommodations and intercity transportation hubs) and inner-city transportation routes for each day.

## Split

- **Default**
- **Easy**: 300 queries with at most one extra constraint.
- **Medium**: 150 queries with complex constraints.
- **Human**: 154 queries produced by humans. Queries in this split are more diverse and may contain unseen constraints in the easy and medium splits.
- **Preference_base50 Split**: 50 base queries used for preference config.

- **Preference**
- **Preference0_base50**: More attractions.
- **Preference1_base50**: Less inner-city transports time.
- **Preference2_base50**: Less average transport time to restaurants.
- **Preference3_base50**: More spending on food.
- **Preference4_base50**: Less spending on accommodation.
- **Preference5_base50**: Shorter distance to \[poi\].

## Record Layout

- "uid": The unique identifier for each query.
- "tag": The tag of the query.
- "start_city": The departure city.
- "target_city": The destination city.
- "days": The number of days for the travel.
- "people_number": The number of people involved in the travel.
- "hard_logic_py": The python codes for the constraints.
- "nature_language": The natural language description or request related to the travel plan.
- "nature_language_en": The English translation of the natural language description.

The keys below are only in preference config:

- "preference": The description of the preference.
- "preference_en": The English translation of the description of the preference.
- "preference_py": The python codes for the preference.

## Citation

If our paper or related resources prove valuable to your research, we kindly ask for citation. Please feel free to contact us with any inquiries.

```bib
@article{Shao2024ChinaTravel,
title={ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning},
author={Jie-Jing Shao and Xiao-Wen Yang and Bo-Wen Zhang and Baizhi Chen and Wen-Da Wei and Guohao Cai and Zhenhua Dong and Lan-Zhe Guo and Yu-feng Li},
year={2024},
journal={arXiv preprint arXiv: 2412.13682},
url={https://arxiv.org/abs/2412.13682},
}
```

# ChinaTravel 数据集
ChinaTravel 是一款悉心设计的基准测试集，旨在为语言智能体（AI Agent）提供全面且可扩展的多日多POI（兴趣点）旅行规划评估框架。详见我们的[论文](https://arxiv.org/pdf/2412.13682)以获取更多细节。

## 简介
在 ChinaTravel 数据集中，针对给定查询，语言智能体需在沙箱环境中使用提供的工具收集信息，并生成JSON格式的旅行规划。该规划需涵盖每日的POI列表（包含餐厅、景点、住宿场所及城际交通枢纽）与城内交通路线。

## 数据集划分
- **默认划分**
- **简单子集**：包含300条查询，最多仅含一项额外约束条件。
- **中等子集**：包含150条查询，约束条件较为复杂。
- **人工子集**：包含154条由人工生成的查询。该子集下的查询多样性更强，可能包含简单、中等子集中未出现的约束条件。
- **Preference_base50 划分**：包含50条基础查询，用于偏好配置。

- **偏好划分**
- **Preference0_base50**：偏好更多景点。
- **Preference1_base50**：偏好更短的城内交通耗时。
- **Preference2_base50**：偏好更短的前往餐厅的平均交通耗时。
- **Preference3_base50**：偏好更高的餐饮支出。
- **Preference4_base50**：偏好更低的住宿支出。
- **Preference5_base50**：偏好到指定POI的更短距离。

## 记录格式
- "uid": 每条查询的唯一标识符。
- "tag": 查询的标签。
- "start_city": 出发城市。
- "target_city": 目的地城市。
- "days": 旅行总天数。
- "people_number": 旅行参与人数。
- "hard_logic_py": 约束条件对应的Python代码。
- "nature_language": 与旅行规划相关的自然语言描述或需求。
- "nature_language_en": 自然语言描述的英文译本。

仅在偏好配置中包含以下键：
- "preference": 偏好设置的描述文本。
- "preference_en": 偏好描述文本的英文译本。
- "preference_py": 偏好配置对应的Python代码。

## 引用说明
若您的研究中用到了本论文或相关资源，恳请您引用本工作。如有任何疑问，欢迎随时与我们联系。

bib
@article{Shao2024ChinaTravel,
title={ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning},
author={Jie-Jing Shao and Xiao-Wen Yang and Bo-Wen Zhang and Baizhi Chen and Wen-Da Wei and Guohao Cai and Zhenhua Dong and Lan-Zhe Guo and Yu-feng Li},
year={2024},
journal={arXiv preprint arXiv: 2412.13682},
url={https://arxiv.org/abs/2412.13682},
}

搜集汇总

背景与挑战

背景概述

ChinaTravel是一个用于评估语言代理在多天多POI旅行规划中表现的综合基准测试数据集。它包含不同难度级别的查询，并提供了详细的记录布局和偏好配置，适用于生成包含POI列表和市内交通路线的旅行计划。