openbmb/UltraInteract_sft

---
language:
- en
license: mit
pretty_name: UltraInteract_sft
configs:
- config_name: default
data_files:
- split: train
path: 0000_sft.parquet
dataset_info:
features:
- name: task
dtype: string
- name: dataset
dtype: string
- name: instruction
dtype: string
- name: response
dtype: string
- name: id
dtype: string
- name: parent_id
dtype: string
splits:
- name: train
num_bytes: 687238
num_examples: 288579
download_size: 687238
dataset_size: 687238
---

## Introduction

- 📜 [Paper](https://arxiv.org/abs/2404.02078)
- 🤗 [Eurus Collection](https://huggingface.co/collections/openbmb/eurus-660bc40bec5376b3adc9d1c5)
- 🤗 UltraInteract
- [SFT](https://huggingface.co/datasets/openbmb/UltraInteract_sft)
- [Preference Learning](https://huggingface.co/datasets/openbmb/UltraInteract_pair)
- [GitHub Repo](https://github.com/OpenBMB/Eurus)

UltraInteract is a large-scale, high-quality alignment dataset specifically designed for complex reasoning tasks. For each instruction, it includes a preference tree consisting of
- (1) reasoning chains with diverse planning strategies in a unified format
- (2) multi-turn interaction trajectories with the environment and the critique
- (3) pairwise data to facilitate preference learning

## Structure
UltraInteract collects a preference tree for each instruction, with the instruction being the root and each action a node. A trajectory is a root-to-leaf path consisting of a sequence of actions. In each preference tree, all nodes of correct actions and all trajectories ending with correct actions can be used for SFT. Paired correct and incorrect nodes or trajectories can be used for preference learning.
<img src="./figures/tree.png" alt="tree" style="zoom: 20%;" />

## Illustrative Example
Here is an illustrative example of an UltraInteract trajectory over two turns. In each turn, the actor model generates step-by-step reasoning chains, and the environment and the critique model provide observations and textual critique respectively.
<img src="./figures/ui_example.png" alt="ui_example" style="zoom: 25%;" />

## Stats
Below are some statistics about UltraInteract. It consists of 86k instructions, 286k correct answers, and 219k pairs.
<img src="./figures/stats.png" alt="stats" style="zoom: 40%;" />

## Dataset Format (SFT)

Coding
```JSONC
{
"id": "92a7356beaa75391539cb79624bdf85b7e5d36d5e74e46a2f1c4001f40b3e055_1_1", // unique data id, contains three parts, the unique string, the depth of the preference tree, and current turn, separated by '_'
"parent_id": "535550938ff58bd73cd7fb85be834bb90b7600daf6b6f6a16b64091e5ecbfe43", // data with the same parent_id have the same instruction but different actions.
"task": "Coding", // task type, choices = ["Coding", "Math_Cot", "Math_PoT", "Logic"]
"dataset": "codecontest", // the dataset where the instruction comes from
"instruction": "Write Python code to solve the task.\nMike has a string s consisting of only lowercase English letters. He wants to change exactly one character from the string so that the resulting one is a palindrome. \n\nA palindrome is a string that reads the same backward as forward, for example strings \"z\", \"aaa\", \"aba\", \"abccba\" are palindromes, but strings \"codeforces\", \"reality\", \"ab\" are not.\n\nInput\n\nThe first and single line contains string s (1 ≤ |s| ≤ 15).\n\nOutput\n\nPrint \"YES\" (without quotes) if Mike can change exactly one character so that the resulting string is palindrome or \"NO\" (without quotes) otherwise. \n\nExamples\n\nInput\n\nabccaa\n\n\nOutput\n\nYES\n\n\nInput\n\nabbcca\n\n\nOutput\n\nNO\n\n\nInput\n\nabcda\n\n\nOutput\n\nYES", // the task description.
"response": "
Step 1: This problem asks us to determine if it is possible to change exactly one character in the string to make it a palindrome.
Step 2: We can start by iterating through the characters in the string and counting the number of differences between corresponding characters from the beginning and end of the string.
Step 3: If the length of the string is odd and there is only one difference, or if the length is even and there is exactly one difference, then it is possible to change exactly one character to make the string a palindrome.
Step 4: We need to consider edge cases such as a single character string or an empty string.

```python
# Step 5: Take the input string
s = input()
# Step 6: Calculate the length of the string
n = len(s)
# Step 7: Initialize a counter to keep track of the number of differences
cnt = 0
# Step 8: Iterate through the first half of the string and count the number of differences between corresponding characters from the beginning and end of the string.
for i in range(n//2):
if s[i] != s[n-i-1]:
cnt += 1
# Step 9: If the length of the string is odd and there are no differences, or if the length is even and there is exactly one difference, print "YES" (without quotes)
if (n % 2 == 1 and cnt == 0) or (cnt == 1):
print('YES')
# Step 10: Otherwise, print "NO" (without quotes)
else:
print('NO')
```" // The step-by-step solution.
}
```

Math
```JSONC
{
"id": "647b2e8706cf459a8e3ec8e99727d2d51ff3414bf187db43d92cff87f8012713_1_1",
"parent_id": "499d250c5852eddcd48a28b6c173f1a2349a16555990a1cd6fb5c7954cb94587",
"task": "Math_PoT",
"dataset": "mathqa",
"instruction": "Tool available:\n[1] Python interpreter\nWhen you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment.\nSolve the following math problem step-by-step.\nthe length of a rectangle is doubled while its width is doubled . what is the % change in area ?",
"response": "
Step 1: The area of a rectangle is given by the product of its length and width. If the length is doubled, the area changes from l*w to 2l*w. Similarly, if the width is doubled, the area changes from l*w to l*2w.
Step 2: If both the length and width are doubled, the area changes from l*w to 2l*2w. The percentage change in area is given by ((new area - old area) / old area)*100.
Step 3: We can calculate the percentage change in area using the formula above.
Step 4: By creating a tool to calculate the percentage change in area, we can easily solve the question.

```python
# Step 5: Define a tool to calculate the percentage change in area
def calculate_percentage_change(length, width):
# Step 6: Calculate the old and new areas
old_area = length * width
new_area = 2 * length * 2 * width
# Step 7: Calculate the percentage change
percentage_change = ((new_area - old_area) / old_area) * 100
return percentage_change
# Step 8: Call the function to solve the question
percentage_change = calculate_percentage_change(2, 2)
print(percentage_change)
```"
}
```

Logic
```JSONC
{
"id": "92a19848da4fb0fc0b414f061f7500a81ea4cbd8f922ca1e32a6a15ef6f7e1a0_1_1",
"parent_id": "107cc9147d8c09e62e2ba9e74cb41d9dbafd9a3adf62b67ba3da2b31c81b3300",
"task": "Logic",
"dataset": "hotpotqa",
"instruction": "
Tool available:\n[1] wikipedia_search(query: str) -> str\nThe Wikipedia Search tool provides access to a vast collection of articles covering a wide range of topics.\nCan query specific keywords or topics to retrieve accurate and comprehensive information.
Solve the following problem step-by-step:\nWhich musician or group is from the US, 3OH!3 or Regina Spektor?
Step 1: To determine which musician or group is from the US, we can search for information about each of them.
```python
# Step 2: Searching for the information about 3OH!3.
print(wikipedia_search('3OH!3'))
```
Page: 3OH!3
Summary: 3OH!3 (pronounced \"three oh three\") is an American electronic music duo from Boulder, Colorado, made up of Sean Foreman and Nathaniel Motte. They are best known for their single \"DONTTRUSTME\" from their album Want, which reached number seven on the Billboard Hot 100. Their second single, a remix of \"Starstrukk\" featuring Katy Perry from Want, was a top ten hit in the United Kingdom, Ireland, Finland, Poland, and Australia. They gained further recognition by featuring Kesha on the song \"My First Kiss\", which was made the lead single from their album Streets of Gold. The album later peaked at number seven on the Billboard 200.\n\n\n\nPage: 3OH!3 (album)\nSummary: 3OH!3 is the debut studio album by American electronic music band 3OH!3. The tracks \"Holler Til You Pass Out\" and \"Chokechain\" were later remixed for the band's major label debut. \"Holler Til You Pass Out\", \"Chokechain\", and \"Dance with Me\" are excluded from the iTunes release of the album.",
"response": "
Step 1: The provided information confirms that 3OH!3 is an American electronic music duo from Boulder, Colorado.
Step 2: We no longer need to perform a search for Regina Spektor as we were only tasked to determine which of the two options is from the US and we have already confirmed 3OH!3's origin.
Step 3: We can confidently conclude that 3OH!3 is the musician/group from the US.
Answer:3OH!3"
}
```

## Citation

```bib
@misc{yuan2024advancing,
title={Advancing LLM Reasoning Generalists with Preference Trees},
author={Lifan Yuan and Ganqu Cui and Hanbin Wang and Ning Ding and Xingyao Wang and Jia Deng and Boji Shan and Huimin Chen and Ruobing Xie and Yankai Lin and Zhenghao Liu and Bowen Zhou and Hao Peng and Zhiyuan Liu and Maosong Sun},
year={2024},
eprint={2404.02078},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```

原始信息汇总

数据集概述

基本信息

• 语言: 英语
• 许可证: MIT
• 名称: UltraInteract_sft
• 配置: 默认配置
  • 数据文件:
    • 分割: 训练
    • 路径: 0000_sft.parquet

数据集特征

• 特征:
  • 名称: task
    • 数据类型: string
  • 名称: dataset
    • 数据类型: string
  • 名称: instruction
    • 数据类型: string
  • 名称: response
    • 数据类型: string
  • 名称: id
    • 数据类型: string
  • 名称: parent_id
    • 数据类型: string

数据分割

• 分割名称: 训练
  • 字节数: 687238
  • 样本数: 288579

数据集大小

• 下载大小: 687238
• 数据集大小: 687238

数据集格式 (SFT)

示例 - Coding

jsonc { "id": "92a7356beaa75391539cb79624bdf85b7e5d36d5e74e46a2f1c4001f40b3e055_1_1", "parent_id": "535550938ff58bd73cd7fb85be834bb90b7600daf6b6f6a16b64091e5ecbfe43", "task": "Coding", "dataset": "codecontest", "instruction": "Write Python code to solve the task. Mike has a string s consisting of only lowercase English letters. He wants to change exactly one character from the string so that the resulting one is a palindrome.

A palindrome is a string that reads the same backward as forward, for example strings "z", "aaa", "aba", "abccba" are palindromes, but strings "codeforces", "reality", "ab" are not.

Input

The first and single line contains string s (1 ≤ |s| ≤ 15).

Output

Print "YES" (without quotes) if Mike can change exactly one character so that the resulting string is palindrome or "NO" (without quotes) otherwise.

Examples

Input

abccaa

Output

YES

Input

abbcca

Output

NO

Input

abcda

Output

YES", "response": " Step 1: This problem asks us to determine if it is possible to change exactly one character in the string to make it a palindrome. Step 2: We can start by iterating through the characters in the string and counting the number of differences between corresponding characters from the beginning and end of the string. Step 3: If the length of the string is odd and there is only one difference, or if the length is even and there is exactly one difference, then it is possible to change exactly one character to make the string a palindrome. Step 4: We need to consider edge cases such as a single character string or an empty string.

    python
    # Step 5: Take the input string
    s = input()
    # Step 6: Calculate the length of the string
    n = len(s)
    # Step 7: Initialize a counter to keep track of the number of differences
    cnt = 0
    # Step 8: Iterate through the first half of the string and count the number of differences between corresponding characters from the beginning and end of the string.
    for i in range(n//2):
        if s[i] != s[n-i-1]:
            cnt += 1
    # Step 9: If the length of the string is odd and there are no differences, or if the length is even and there is exactly one difference, print "YES" (without quotes)
    if (n % 2 == 1 and cnt == 0) or (cnt == 1):
        print(YES)
    # Step 10: Otherwise, print "NO" (without quotes)
    else:
        print(NO)
    "

}

示例 - Math

jsonc { "id": "647b2e8706cf459a8e3ec8e99727d2d51ff3414bf187db43d92cff87f8012713_1_1", "parent_id": "499d250c5852eddcd48a28b6c173f1a2349a16555990a1cd6fb5c7954cb94587", "task": "Math_PoT", "dataset": "mathqa", "instruction": "Tool available: [1] Python interpreter When you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment. Solve the following math problem step-by-step. the length of a rectangle is doubled while its width is doubled . what is the % change in area ?", "response": " Step 1: The area of a rectangle is given by the product of its length and width. If the length is doubled, the area changes from lw to 2lw. Similarly, if the width is doubled, the area changes from lw to l2w. Step 2: If both the length and width are doubled, the area changes from lw to 2l2w. The percentage change in area is given by ((new area - old area) / old area)*100. Step 3: We can calculate the percentage change in area using the formula above. Step 4: By creating a tool to calculate the percentage change in area, we can easily solve the question.

    python
    # Step 5: Define a tool to calculate the percentage change in area
    def calculate_percentage_change(length, width):
        # Step 6: Calculate the old and new areas
        old_area = length * width
        new_area = 2 * length * 2 * width
        # Step 7: Calculate the percentage change
        percentage_change = ((new_area - old_area) / old_area) * 100
        return percentage_change
    # Step 8: Call the function to solve the question
    percentage_change = calculate_percentage_change(2, 2)
    print(percentage_change)
    "

}

示例 - Logic

jsonc { "id": "92a19848da4fb0fc0b414f061f7500a81ea4cbd8f922ca1e32a6a15ef6f7e1a0_1_1", "parent_id": "107cc9147d8c09e62e2ba9e74cb41d9dbafd9a3adf62b67ba3da2b31c81b3300", "task": "Logic", "dataset": "hotpotqa", "instruction": " Tool available: [1] wikipedia_search(query: str) -> str The Wikipedia Search tool provides access to a vast collection of articles covering a wide range of topics. Can query specific keywords or topics to retrieve accurate and comprehensive information. Solve the following problem step-by-step: Which musician or group is from the US, 3OH!3 or Regina Spektor? Step 1: To determine which musician or group is from the US, we can search for information about each of them. python # Step 2: Searching for the information about 3OH!3. print(wikipedia_search(3OH!3))

    Page: 3OH!3
    Summary: 3OH!3 (pronounced "three oh three") is an American electronic music duo from Boulder, Colorado, made up of Sean Foreman and Nathaniel Motte. They are best known for their single "DONTTRUSTME" from their album Want, which reached number seven on the Billboard Hot 100. Their second single, a remix of "Starstrukk" featuring Katy Perry from Want, was a top ten hit in the United Kingdom, Ireland, Finland, Poland, and Australia. They gained further recognition by featuring Kesha on the song "My First Kiss", which was made the lead single from their album Streets of Gold. The album later peaked at number seven on the Billboard 200.

Page: 3OH!3 (album) Summary: 3OH!3 is the debut studio album by American electronic music band 3OH!3. The tracks "Holler Til You Pass Out" and "Chokechain" were later remixed for the bands major label debut. "Holler Til You Pass Out", "Chokechain", and "Dance with Me" are excluded from the iTunes release of the album.", "response": " Step 1: The provided information confirms that 3OH!3 is an American electronic music duo from Boulder, Colorado. Step 2: We no longer need to perform a search for Regina Spektor as we were only tasked to determine which of the two options is from the US and we have already confirmed 3OH!3s origin. Step 3: We can confidently conclude that 3OH!3 is the musician/group from the US. Answer:3OH!3" }

搜集汇总

数据集介绍

构建方式

UltraInteract_sft 数据集由 OpenBMB 团队精心构建，旨在服务于复杂推理任务的大规模高质量对齐。其核心构建方式围绕“偏好树”结构展开：每条指令作为根节点，每个推理或交互动作作为子节点，形成从根到叶的完整轨迹。数据集中，所有正确动作的节点以及以正确动作结束的轨迹均被筛选出来，用于监督式微调（SFT）。构建过程覆盖了编码、数学推理（CoT与PoT）以及逻辑推理等多种任务类型，确保了数据的多样性与复杂性。通过从86k条指令中提取286k个正确答案，该数据集为强化模型在复杂场景下的推理能力提供了坚实的训练基础。

使用方法

使用 UltraInteract_sft 数据集时，用户可直接加载 HuggingFace 上的 parquet 文件，通过指定 'train' 分割获取全部288,579条样本。每条样本包含 'instruction' 与 'response' 字段，可直接用于序列到序列的监督式微调。数据集中 'task' 字段标识任务类型，便于按需筛选特定领域数据；'parent_id' 字段则支持关联同一指令下的不同动作，适用于对比学习或偏好对齐实验。用户可基于 'id' 字段追踪数据在偏好树中的层级与轮次，从而灵活设计训练策略，如逐步推理或交互式学习。

背景与挑战

背景概述

UltraInteract_sft是由清华大学自然语言处理实验室（THUNLP）与OpenBMB团队于2024年联合发布的大规模对齐数据集，旨在提升大语言模型在复杂推理任务上的泛化能力。该数据集基于偏好树结构，为每条指令构建了包含多样化规划策略的推理链、多轮交互轨迹及偏好对，覆盖代码生成、数学推理与逻辑问答等核心领域。其研究问题聚焦于如何通过结构化对齐数据，使模型在无需显式搜索的情况下，掌握多步推理与自我纠错能力。作为Eurus系列的核心组件，UltraInteract_sft为后续的偏好学习提供了高质量监督信号，对推动推理导向的AI对齐研究具有里程碑意义。

当前挑战

当前数据集面临的核心挑战在于复杂推理任务的领域泛化性。尽管UltraInteract_sft通过偏好树结构覆盖了代码、数学与逻辑任务，但真实场景中推理链条的多样性（如多步数学证明、开放式编程调试）仍难以被有限模板穷举，导致模型在未见任务类型上易产生逻辑断裂。构建过程中，如何自动生成高保真度的环境反馈与批判性评价是一大瓶颈——依赖人工标注成本过高，而基于规则或弱监督的自动标注又可能引入噪声，污染偏好树的正确性。此外，多轮交互轨迹中“正确动作”与“错误路径”的边界模糊，进一步增加了偏好学习阶段的判别难度，亟需更精细的节点质量筛选机制。

常用场景

经典使用场景

UltraInteract_sft数据集专为复杂推理任务的大规模高质量对齐设计，其经典使用场景在于构建和微调大语言模型的监督式微调（SFT）阶段。通过提供包含多样化规划策略的推理链、多轮交互轨迹以及环境与批判反馈的结构化偏好树，该数据集使模型能够学习到从编程、数学到逻辑推理等多种任务中逐步求解的范式，显著提升模型在复杂场景下的推理能力与鲁棒性。

解决学术问题

该数据集有效解决了大语言模型在复杂推理任务中对齐训练中数据质量与多样性不足的学术难题。传统对齐方法常受限于单一指令或简单问答，难以覆盖推理过程中的策略多样性与交互深度。UltraInteract_sft通过引入偏好树结构，融合正确与错误轨迹的对比数据，为研究模型如何从多步推理、错误修正及环境反馈中学习提供了系统化的数据基础，推动了推理泛化能力的理论探索。

实际应用

在实际应用中，UltraInteract_sft可用于开发具备高阶推理能力的智能助手，例如自动编程调试、数学问题求解和逻辑问答系统。基于该数据集训练的模型能够处理需要多步规划和环境交互的复杂任务，如代码生成与验证、科学计算推理，以及多源信息整合的决策支持，从而在教育和软件开发等领域实现更精准、可靠的自动化推理服务。

数据集最近研究

最新研究方向

在大语言模型复杂推理能力对齐的前沿探索中，UltraInteract_sft数据集以其独特的偏好树结构，为模型训练提供了涵盖编码、数学与逻辑推理的多轮交互轨迹。该数据集不仅整合了多样化的规划策略与批判性反馈，还通过配对的正负样本推动偏好学习，成为提升LLM推理泛化能力的关键资源。其设计紧密关联当前AI领域对可解释、可验证推理路径的迫切需求，为构建更具鲁棒性的通用推理智能体奠定了数据基础，代表了从静态指令微调向动态交互对齐的重要演进方向。