tuandunghcmut/toolbench-v1

---
license: apache-2.0
task_categories:
- question-answering
- text-generation
language:
- en
tags:
- tool-use
- function-calling
- toolbench
size_categories:
- 100K<n<1M
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
- config_name: benchmark
data_files:
- split: g1_instruction
path: benchmark/g1_instruction-*
- split: g1_category
path: benchmark/g1_category-*
- split: g1_tool
path: benchmark/g1_tool-*
- split: g2_instruction
path: benchmark/g2_instruction-*
- split: g2_category
path: benchmark/g2_category-*
- split: g3_instruction
path: benchmark/g3_instruction-*
---

# ToolBench Dataset

## Dataset Description

**ToolBench** is an open-source, large-scale, high-quality instruction tuning SFT dataset designed to facilitate the construction of powerful LLMs with general tool-use capability. It was constructed automatically using ChatGPT (gpt-3.5-turbo-16k) upgraded with enhanced function call capabilities.

This dataset corresponds to the training data used for **ToolLLaMA**.

- **Repository:** [OpenBMB/ToolBench](https://github.com/OpenBMB/ToolBench)
- **Paper:** [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://arxiv.org/abs/2307.16789)

## Dataset Structure

The dataset contains conversations between a user and an assistant (AutoGPT/ToolLLaMA) involving tool calls.

### Data Fields

- `id`: Unique identifier for the conversation.
- `conversations`: A list of messages between the "user", "system", and "assistant" (or "function").

### Example

```json
{
"id": "12345",
"conversations": [
{
"from": "system",
"value": "You are AutoGPT..."
},
{
"from": "user",
"value": "I need to find..."
},
...
]
}
```

## Dataset Configuration

The dataset is available in two configurations:

### 1. Default (`default`)
Contains the conversation data for training and validation.
- **Splits**: `train`, `validation`
- **Schema**: `id`, `conversations`

```python
from datasets import load_dataset
ds = load_dataset("tuandunghcmut/toolbench-v1", "default")
```

### 2. Benchmark (`benchmark`)
Contains the test instructions for evaluation.
- **Splits**: `g1_instruction`, `g1_category`, `g1_tool`, `g2_instruction`, `g2_category`, `g3_instruction`
- **Schema**: `query_id`, `query`, `api_list` (JSON string), `relevant_apis` (JSON string)

```python
from datasets import load_dataset
ds = load_dataset("tuandunghcmut/toolbench-v1", "benchmark")
```

## Citation

```bibtex
@misc{qin2023toolllm,
title={ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs},
author={Yujia Qin and Shihao Liang and Yining Ye and Kunlun Zhu and Lan Yan and Yaxi Lu and Yankai Lin and Xin Cong and Xiangru Tang and Bill Qian and Sihan Zhao and Runchu Tian and Ruobing Xie and Jie Zhou and Mark Gerstein and Dahai Li and Zhiyuan Liu and Maosong Sun},
year={2023},
eprint={2307.16789},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```

---
许可证: Apache-2.0
任务类别:
- 问答
- 文本生成
语言:
- 英语
标签:
- 工具使用
- 函数调用
- ToolBench
样本规模:
- 100K < n < 1M
配置项:
- 配置名称: default
数据文件:
- 拆分集: train
路径: data/train-*
- 拆分集: validation
路径: data/validation-*
- 配置名称: benchmark
数据文件:
- 拆分集: g1_instruction
路径: benchmark/g1_instruction-*
- 拆分集: g1_category
路径: benchmark/g1_category-*
- 拆分集: g1_tool
路径: benchmark/g1_tool-*
- 拆分集: g2_instruction
路径: benchmark/g2_instruction-*
- 拆分集: g2_category
路径: benchmark/g2_category-*
- 拆分集: g3_instruction
路径: benchmark/g3_instruction-*
---
# ToolBench 数据集
## 数据集描述
**ToolBench** 是一款开源、大规模、高质量的面向指令微调的监督微调（Supervised Fine-Tuning, SFT）数据集，旨在助力构建具备通用工具使用能力的强大大语言模型（Large Language Model, LLM）。该数据集通过搭载增强函数调用能力的ChatGPT（gpt-3.5-turbo-16k）自动构建完成。
本数据集对应 **ToolLLaMA** 所使用的训练数据。
- **仓库地址**: [OpenBMB/ToolBench](https://github.com/OpenBMB/ToolBench)
- **论文链接**: [ToolLLM: 助力大语言模型掌握16000+真实世界API](https://arxiv.org/abs/2307.16789)

## 数据集结构
本数据集包含用户与助手（AutoGPT/ToolLLaMA）之间涉及工具调用的对话数据。
### 数据字段
- `id`: 对话的唯一标识符。
- `conversations`: 包含“用户（user）”“系统（system）”与“助手（assistant）”（或“函数（function）”）之间的消息列表。

### 示例
json
{
"id": "12345",
"conversations": [
{
"from": "system",
"value": "You are AutoGPT..."
},
{
"from": "user",
"value": "I need to find..."
},
...
]
}

## 数据集配置
本数据集提供两种配置方案：
### 1. 默认配置（`default`）
包含用于训练与验证的对话数据。
- **拆分集**: `train`、`validation`
- **数据 Schema**: `id`、`conversations`
python
from datasets import load_dataset
ds = load_dataset("tuandunghcmut/toolbench-v1", "default")

### 2. 基准测试配置（`benchmark`）
包含用于评估的测试指令数据。
- **拆分集**: `g1_instruction`、`g1_category`、`g1_tool`、`g2_instruction`、`g2_category`、`g3_instruction`
- **数据 Schema**: `query_id`、`query`、`api_list`（JSON字符串）、`relevant_apis`（JSON字符串）
python
from datasets import load_dataset
ds = load_dataset("tuandunghcmut/toolbench-v1", "benchmark")

## 引用
bibtex
@misc{qin2023toolllm,
title={ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs},
author={Yujia Qin and Shihao Liang and Yining Ye and Kunlun Zhu and Lan Yan and Yaxi Lu and Yankai Lin and Xin Cong and Xiangru Tang and Bill Qian and Sihan Zhao and Runchu Tian and Ruobing Xie and Jie Zhou and Mark Gerstein and Dahai Li and Zhiyuan Liu and Maosong Sun},
year={2023},
eprint={2307.16789},
archivePrefix={arXiv},
primaryClass={cs.AI}
}