tuandunghcmut/toolbench-v1

--- license: apache-2.0 task_categories: - question-answering - text-generation language: - en tags: - tool-use - function-calling - toolbench size_categories: - 100K<n<1M configs: - config_name: default data_files: - split: train path: data/train-* - split: validation path: data/validation-* - config_name: benchmark data_files: - split: g1_instruction path: benchmark/g1_instruction-* - split: g1_category path: benchmark/g1_category-* - split: g1_tool path: benchmark/g1_tool-* - split: g2_instruction path: benchmark/g2_instruction-* - split: g2_category path: benchmark/g2_category-* - split: g3_instruction path: benchmark/g3_instruction-* --- # ToolBench Dataset ## Dataset Description **ToolBench** is an open-source, large-scale, high-quality instruction tuning SFT dataset designed to facilitate the construction of powerful LLMs with general tool-use capability. It was constructed automatically using ChatGPT (gpt-3.5-turbo-16k) upgraded with enhanced function call capabilities. This dataset corresponds to the training data used for **ToolLLaMA**. - **Repository:** [OpenBMB/ToolBench](https://github.com/OpenBMB/ToolBench) - **Paper:** [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://arxiv.org/abs/2307.16789) ## Dataset Structure The dataset contains conversations between a user and an assistant (AutoGPT/ToolLLaMA) involving tool calls. ### Data Fields - `id`: Unique identifier for the conversation. - `conversations`: A list of messages between the "user", "system", and "assistant" (or "function"). ### Example ```json { "id": "12345", "conversations": [ { "from": "system", "value": "You are AutoGPT..." }, { "from": "user", "value": "I need to find..." }, ... ] } ``` ## Dataset Configuration The dataset is available in two configurations: ### 1. Default (`default`) Contains the conversation data for training and validation. - **Splits**: `train`, `validation` - **Schema**: `id`, `conversations` ```python from datasets import load_dataset ds = load_dataset("tuandunghcmut/toolbench-v1", "default") ``` ### 2. Benchmark (`benchmark`) Contains the test instructions for evaluation. - **Splits**: `g1_instruction`, `g1_category`, `g1_tool`, `g2_instruction`, `g2_category`, `g3_instruction` - **Schema**: `query_id`, `query`, `api_list` (JSON string), `relevant_apis` (JSON string) ```python from datasets import load_dataset ds = load_dataset("tuandunghcmut/toolbench-v1", "benchmark") ``` ## Citation ```bibtex @misc{qin2023toolllm, title={ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs}, author={Yujia Qin and Shihao Liang and Yining Ye and Kunlun Zhu and Lan Yan and Yaxi Lu and Yankai Lin and Xin Cong and Xiangru Tang and Bill Qian and Sihan Zhao and Runchu Tian and Ruobing Xie and Jie Zhou and Mark Gerstein and Dahai Li and Zhiyuan Liu and Maosong Sun}, year={2023}, eprint={2307.16789}, archivePrefix={arXiv}, primaryClass={cs.AI} } ```--- 许可证: Apache-2.0 任务类别: - 问答 - 文本生成 语言: - 英语 标签: - 工具使用 - 函数调用 - ToolBench 样本规模: - 100K < n < 1M 配置项: - 配置名称: default 数据文件: - 拆分集: train 路径: data/train-* - 拆分集: validation 路径: data/validation-* - 配置名称: benchmark 数据文件: - 拆分集: g1_instruction 路径: benchmark/g1_instruction-* - 拆分集: g1_category 路径: benchmark/g1_category-* - 拆分集: g1_tool 路径: benchmark/g1_tool-* - 拆分集: g2_instruction 路径: benchmark/g2_instruction-* - 拆分集: g2_category 路径: benchmark/g2_category-* - 拆分集: g3_instruction 路径: benchmark/g3_instruction-* --- # ToolBench 数据集 ## 数据集描述 **ToolBench** 是一款开源、大规模、高质量的面向指令微调的监督微调（Supervised Fine-Tuning, SFT）数据集，旨在助力构建具备通用工具使用能力的强大大语言模型（Large Language Model, LLM）。该数据集通过搭载增强函数调用能力的ChatGPT（gpt-3.5-turbo-16k）自动构建完成。 本数据集对应 **ToolLLaMA** 所使用的训练数据。 - **仓库地址**: [OpenBMB/ToolBench](https://github.com/OpenBMB/ToolBench) - **论文链接**: [ToolLLM: 助力大语言模型掌握16000+真实世界API](https://arxiv.org/abs/2307.16789) ## 数据集结构 本数据集包含用户与助手（AutoGPT/ToolLLaMA）之间涉及工具调用的对话数据。 ### 数据字段 - `id`: 对话的唯一标识符。 - `conversations`: 包含“用户（user）”“系统（system）”与“助手（assistant）”（或“函数（function）”）之间的消息列表。 ### 示例 json { "id": "12345", "conversations": [ { "from": "system", "value": "You are AutoGPT..." }, { "from": "user", "value": "I need to find..." }, ... ] } ## 数据集配置 本数据集提供两种配置方案： ### 1. 默认配置（`default`） 包含用于训练与验证的对话数据。 - **拆分集**: `train`、`validation` - **数据 Schema**: `id`、`conversations` python from datasets import load_dataset ds = load_dataset("tuandunghcmut/toolbench-v1", "default") ### 2. 基准测试配置（`benchmark`） 包含用于评估的测试指令数据。 - **拆分集**: `g1_instruction`、`g1_category`、`g1_tool`、`g2_instruction`、`g2_category`、`g3_instruction` - **数据 Schema**: `query_id`、`query`、`api_list`（JSON字符串）、`relevant_apis`（JSON字符串） python from datasets import load_dataset ds = load_dataset("tuandunghcmut/toolbench-v1", "benchmark") ## 引用 bibtex @misc{qin2023toolllm, title={ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs}, author={Yujia Qin and Shihao Liang and Yining Ye and Kunlun Zhu and Lan Yan and Yaxi Lu and Yankai Lin and Xin Cong and Xiangru Tang and Bill Qian and Sihan Zhao and Runchu Tian and Ruobing Xie and Jie Zhou and Mark Gerstein and Dahai Li and Zhiyuan Liu and Maosong Sun}, year={2023}, eprint={2307.16789}, archivePrefix={arXiv}, primaryClass={cs.AI} }

搜集汇总

背景与挑战

背景概述

ToolBench是一个开源的大规模高质量指令调优SFT数据集，旨在通过自动生成的对话数据（使用增强函数调用能力的ChatGPT构建）来训练大型语言模型掌握工具使用能力。该数据集包含用户与助手之间的多轮对话，涉及工具调用，适用于问答和文本生成任务，主要用于支持ToolLLaMA等模型的训练和评估。