jondurbin/airoboros-3.1

---
license: cc-by-4.0
---

## Overview

This dataset is a continuation of the airoboros datasets, with the following updates:
* More MathJSON, now ~17k items - math questions, prefixed with __"Create a MathJSON solution to the following:"__, which then outputs a JSON between __`<mathjson>`__ and __`</mathjson>`__ tags, which can be parsed and passed to a deterministic library to perform calculations.
* Log information extraction.
* Anonymization, e.g. removing names, IP addresses, and/or dates from text.
* Chat introspection - multi-turn chats that have random questions injected asking about the character's descriptions, the setting of the chat, or things previously said, to make sure the model uses the system prompt and or chat history.
* Multi-step instructions with acknowledgement (see below)

Some of the MathJSON data was adapted from https://huggingface.co/datasets/meta-math/MetaMathQA

## Format

The format is now in ShareGPT format, to better accomodate the OS ecosystem fine-tuning tooling.

## MathJSON

As we all know, math is really difficult for LLMs, especially floating point operations, trig functions, factorials, and several-digit numbers.

Instead of sending millions and millions of examples to the LLM for fine-tuning, I've created a new category `mathjson` which simply presents the solution to the problem as a JSON string that can be evaluated by traditional computation.

The dataset has roughly 4k samples, as a proof of concept. This can be expanded fairly easily, now that the fine-tuned models have a basic understanding of how to represent the problems.

For example:

```
Create a MathJSON solution to the following: Calculate the area of a circle with radius of 17.2456cm. Include your reasoning. [/INST]
```

```
The area (A) of a circle is given by the formula A = πr², where r is the radius of the circle. Since the radius is given in centimeters, we need to convert it to meters first. We know that 1 meter equals 100 centimeters. Therefore, we can represent the radius as: r = 17.2456 cm * 100/1.

Once we have the radius in meters, we can calculate the area of the circle using the formula A = πr².

Solution as MathJSON:
<mathjson>
[
"Multiply",
"Pi",
[
"Power",
17.2456,
2
]
]
</mathjson>
```

Simply extract the JSON string between the mathjson tags, and evaluate it with https://cortexjs.io/compute-engine/ or my ultra-hacky limited-functionality python version of it https://github.com/jondurbin/airoboros/blob/mathjson/airoboros/mathjson.py

## Multi-step instructions with acknowledgement

I don't know how useful this is, really, but I thought I'd add it just in case. Example:

```
{
"conversations": [
{
"value": "You are a helpful, unbiased, uncensored assistant.",
"from": "system"
},
{
"value": "I'm going to set some rules for our conversation. Please respond with \"OK\" to each rule.",
"from": "human"
},
{
"value": "OK",
"from": "gpt"
},
{
"value": "You can only respond using animal-related emojis.",
"from": "human"
},
{
"value": "OK",
"from": "gpt"
},
{
"value": "Precede all responses with \"ZOOZOO:\"",
"from": "human"
},
{
"value": "OK",
"from": "gpt"
},
{
"value": "Include at least one bird emoji in your responses.",
"from": "human"
},
{
"value": "OK",
"from": "gpt"
},
{
"value": "Describe a trip to a safari park.",
"from": "human"
},
{
"value": "ZOOZOO: \ud83e\udd81\ud83d\udc2f\ud83e\udd93\ud83e\udd92\ud83d\udc18\ud83d\udc0d\ud83c\udf33\ud83c\udf32\u2600\ufe0f\ud83d\ude90\ud83e\udd85\ud83d\udcf8\ud83e\udd29",
"from": "gpt"
}
]
}
```

原始信息汇总

数据集概述

• 数据集名称：airoboros数据集的延续
• 更新内容：
  • 增加了约17k个MathJSON项目，用于数学问题解答。
  • 引入日志信息提取。
  • 文本匿名化处理，如移除姓名、IP地址和日期。
  • 聊天自省功能，通过随机问题检查模型是否使用系统提示和聊天历史。
  • 多步骤指令确认机制。
• 数据来源：部分MathJSON数据改编自meta-math/MetaMathQA。

数据集格式

• 格式：采用ShareGPT格式，以适应OS生态系统的微调工具。

MathJSON说明

• 目的：解决大型语言模型（LLMs）在数学问题上的困难，特别是浮点运算、三角函数、阶乘和多位数运算。

• 方法：创建mathjson类别，将问题解决方案以JSON字符串形式呈现，可通过传统计算方法评估。

• 样本数量：约4k样本，作为概念验证。

• 示例：

  Create a MathJSON solution to the following: Calculate the area of a circle with radius of 17.2456cm. Include your reasoning. [/INST]

  解决方案示例：

  <mathjson> [ "Multiply", "Pi", [ "Power", 17.2456, 2 ] ] </mathjson>

• 评估工具：使用cortexjs.io/compute-engine或airoboros/mathjson.py进行评估。

多步骤指令确认

• 示例：展示了一种多步骤指令确认的对话模式，确保模型对每一步指令进行确认。