toolverifier

# TOOLVERIFIER: Generalization to New Tools via Self-Verification

This repository contains the ToolSelect dataset which was used to fine-tune Llama-2 70B for tool selection.

## Data

**ToolSelect** data is synthetic training data generated for tool selection task using Llama-2 70B and Llama-2-Chat-70B.
It consists of 555 samples corresponding to 173 tools.
Each training sample is composed of a user instruction, a candidate set of tools that includes the
ground truth tool, and a reasoning note elucidating the correct choice of tool.
For example,
```
User: 10 reviews for "Harry Potter and the Philosopher's Stone".
###
Tool Choices: Bank Account Number generator = The Bank Account Number tool generates a random bank account number for a bank.
Train Ticket Purchase = The Train Ticket Purchase tool gets train tickets for a train, provided stations, and date information.
The Cricket Score tool = The Cricket Score tool gets the score for a match that is happening at a given location and date.
Book Review = The Book Review tool gets the top-rated book reviews for a particular book.
Grocery Shopping with discounts = The Grocery Shopping tool calculates the expense of grocery shopping based on the menu.
Stock Price at location and date = The Stock Price tool gets the stock price for a company at a given location and date.
Movie Recommendation = The Movie Recommendation tool recommends movies based on the user's preferences.
News = The News tool gets the top news for a particular topic or query.
###
Thought: I recommended the Book Review tool because the user asked for reviews of a specific book, "Harry Potter and the Philosopher's Stone". The Book Review tool is designed to provide top-rated book reviews for a particular book, making it the most suitable tool for this request. It will retrieve the top 10 reviews for the book, as requested by the user.
Act: CALLTOOL["Book Review"]
```

### Files
The `data/` folder has 2 files:
* `train.csv` - this file contains the training samples.
* `tools.csv` - this file contains names and descriptions of the generated synthetic tools.

To learn more about the data generation procedure, we direct readers to section 2.1 of our paper.

Paper: https://arxiv.org/abs/2402.14158

## Citation
```
@article{mekala2024toolverifier,
title={TOOLVERIFIER: Generalization to New Tools via Self-Verification},
author={Mekala, Dheeraj and Weston, Jason and Lanchantin, Jack and Raileanu, Roberta and Lomeli, Maria and Shang, Jingbo and Dwivedi-Yu, Jane},
journal={arXiv preprint arXiv:2402.14158},
year={2024}
}
```
## Licensing
See our LICENSE file for licensing details.

# TOOLVERIFIER：基于自验证实现新工具泛化

本仓库包含用于微调Llama-2 70B以完成工具选择任务的ToolSelect数据集。

## 数据

**ToolSelect** 数据是使用Llama-2 70B与Llama-2-Chat-70B为工具选择任务生成的合成训练数据。该数据集包含对应173个工具的555条样本。每条训练样本由用户指令、包含真实标注工具的候选工具集，以及阐明正确工具选择的推理说明三部分组成。示例如下：

用户：查询《哈利·波特与魔法石》的10条评论。
###
工具选择：
银行账户号码生成器 = 银行账户号码生成器工具可随机生成银行账户号码。
火车票购买工具 = 火车票购买工具可根据提供的车站与日期信息购买火车票。
板球比分工具 = 板球比分工具可获取指定地点与日期的赛事比分。
图书评论工具 = 图书评论工具可获取特定书籍的高评分评论。
带折扣的杂货采购工具 = 杂货采购工具可根据购物清单计算购物开销。
指定地点与日期的股票价格工具 = 股票价格工具可获取指定地点与日期的公司股票价格。
电影推荐工具 = 电影推荐工具可根据用户偏好推荐电影。
新闻工具 = 新闻工具可获取特定主题或查询的热门新闻。
###
推理过程：我推荐图书评论工具，因为用户询问的是特定书籍《哈利·波特与魔法石》的评论。图书评论工具专为特定书籍提供高评分评论而设计，是适配该请求的最优工具。它将如用户所求，检索该书籍的前10条评论。
动作：CALLTOOL["图书评论"]

### 文件

`data/` 文件夹包含2个文件：
* `train.csv`：该文件存储全部训练样本。
* `tools.csv`：该文件存储生成的合成工具的名称与功能描述。

若需了解更多数据生成流程，请参阅论文的2.1章节。
论文链接：https://arxiv.org/abs/2402.14158

## 引用

@article{mekala2024toolverifier,
title={TOOLVERIFIER：基于自验证实现新工具泛化},
author={Mekala, Dheeraj and Weston, Jason and Lanchantin, Jack and Raileanu, Roberta and Lomeli, Maria and Shang, Jingbo and Dwivedi-Yu, Jane},
journal={arXiv preprint arXiv:2402.14158},
year={2024}
}

## 许可

详细许可信息请参阅仓库内的LICENSE文件。