One-Shot-CFT-Data

# One-Shot-CFT: Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem

<p align="center">
<a href="https://github.com/TIGER-AI-Lab/One-Shot-CFT" target="_blank">💻 Code</a> |
<a href="https://arxiv.org/pdf/2506.03295" target="_blank">📄 Paper</a> |
<a href="https://huggingface.co/datasets/TIGER-Lab/One-Shot-CFT-Data" target="_blank">📊 Dataset</a> |
<a href="https://huggingface.co/collections/TIGER-Lab/one-shot-cft-683fbb4d2bcf698dbea8fb21" target="_blank">🤗 Model</a> |
<a href="https://tiger-ai-lab.github.io/One-Shot-CFT/" target="_blank">🌐 Project Page</a>
</p>

## 🧠 Overview

One-Shot Critique Fine-Tuning (CFT) is a simple, robust, and compute-efficient training paradigm for unleashing the reasoning capabilities of pretrained LLMs in both mathematical and logical domains. By leveraging critiques on just one problem, One-Shot CFT enables models like Qwen and LLaMA to match or even outperform reinforcement learning, while using 20× less compute.

Instead of learning from reference answers (as in supervised fine-tuning) or reward signals (as in reinforcement learning), One-Shot CFT enables models to learn from critiques of diverse solutions to a single problem, enhancing their exposure to varied reasoning patterns and mitigating overfitting. This exposes the LLMs to multiple perspectives and error types, thereby more effectively unleashing their reasoning potential.

## ✨ Key Highlights

- **Unleashes Reasoning with One Example:** One-Shot CFT uses critiques of diverse model-generated solutions to a single problem to significantly boost performance across math and logic tasks. For example, with just 5 GPU hours of training on Qwen2.5-Math-7B, One-Shot CFT achieves an average improvement of +15% on six math benchmarks and +16% on three logic reasoning benchmarks.
- **Outperforms RLVR and Full SFT with 20× Less Compute:** One-Shot CFT outperforms both one-shot Reinforcement Learning with Verifiable Rewards (RLVR) and full-dataset supervised fine-tuning, while requiring only 5 GPU hours on a 7B model—offering a much more efficient and stable training alternative.
- **Robust Across Seeds and Model Scales:** One-Shot CFT remains effective across different seed problem choices and model sizes—from 1.5B to 14B parameters—demonstrating strong generalization and scalability.

**In this data repository, you can find the One-Shot-CFT training data generated from four questions selected from DeepScaleR, as mentioned in the paper, as well as the One-Shot-CFT training data generated from the corresponding questions of the three tasks — CausalUnderstanding, DisambiguationQA, and TimeArithmetic — in BigBench Extra Hard.**

## Main Results

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/636a35eff8d9af4aea181608/DCxRSdeDrv-Db4VLuEl0T.png" alt="CFT Performance Comparison" width="1100"/>
</p>

<p align="center"><em>
One-shot CFT consistently improves mathematical and logical reasoning.
<strong>Left:</strong> Average accuracy on six mathematical reasoning benchmarks for Qwen and LLaMA models, comparing base, SFT, RLVR, and CFT with only one training example.
<strong>Right:</strong> In-domain accuracy on three logic reasoning benchmarks (BBEH subtasks) for Qwen2.5-Math-7B.
Across both domains, CFT with a single problem significantly outperforms standard SFT and matches or exceeds reinforcement learning with much lower compute.
</em></p>

## Citation

If you find our work helpful, please cite it as:

```bibtex
@article{wang2025unleashing,
title={Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem},
author={Wang, Yubo and Nie, Ping and Zou, Kai and Wu, Lijun and Chen, Wenhu},
journal={arXiv preprint arXiv:2506.03295},
year={2025}
}
```

# 单样本批判微调（One-Shot-CFT）：通过单问题批判微调激发预训练大语言模型的推理潜能

<p align="center">
<a href="https://github.com/TIGER-AI-Lab/One-Shot-CFT" target="_blank">💻 代码</a> |
<a href="https://arxiv.org/pdf/2506.03295" target="_blank">📄 论文</a> |
<a href="https://huggingface.co/datasets/TIGER-Lab/One-Shot-CFT-Data" target="_blank">📊 数据集</a> |
<a href="https://huggingface.co/collections/TIGER-Lab/one-shot-cft-683fbb4d2bcf698dbea8fb21" target="_blank">🤗 模型</a> |
<a href="https://tiger-ai-lab.github.io/One-Shot-CFT/" target="_blank">🌐 项目主页</a>
</p>

## 🧠 概述

单样本批判微调（One-Shot Critique Fine-Tuning，简称CFT）是一种简洁、鲁棒且计算高效的训练范式，旨在激发预训练大语言模型（LLM）在数学与逻辑领域的推理能力。仅通过单个问题的批判反馈，单样本CFT即可让Qwen、LLaMA等模型达到甚至超越强化学习的性能表现，同时计算量仅为后者的1/20。

与监督微调（Supervised Fine-Tuning，SFT）中从参考答案学习、强化学习中从奖励信号学习不同，单样本CFT让模型从单个问题的多样化解决方案的批判反馈中学习，增强模型对多样推理模式的接触，并缓解过拟合。这一机制让大语言模型接触到多种视角与错误类型，从而更高效地激发其推理潜能。

## ✨ 核心亮点

- **单样本即可激发推理能力：** 单样本CFT通过对单个问题的多种模型生成解决方案进行批判反馈，显著提升模型在数学与逻辑任务上的性能。例如，在Qwen2.5-Math-7B模型上仅需5 GPU小时的训练，单样本CFT即可在6个数学基准测试中实现平均15%的性能提升，在3个逻辑推理基准测试中实现平均16%的性能提升。
- **以1/20的计算量超越RLVR与全量SFT：** 单样本CFT的性能优于单样本可验证奖励强化学习（Reinforcement Learning with Verifiable Rewards，RLVR）与全数据集监督微调，且在7B参数模型上仅需5 GPU小时的训练，为训练提供了更为高效且稳定的替代方案。
- **跨种子与模型规模均表现鲁棒：** 单样本CFT在不同种子问题选择与不同模型规模（从1.5B到14B参数）下均能保持有效，展现出极强的泛化性与可扩展性。

**在本数据仓库中，您可找到论文中提及的从DeepScaleR中选取的4个问题生成的单样本CFT训练数据，以及从BigBench Extra Hard中3个任务（CausalUnderstanding、DisambiguationQA与TimeArithmetic）的对应问题生成的单样本CFT训练数据。**

## 主要实验结果

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/636a35eff8d9af4aea181608/DCxRSdeDrv-Db4VLuEl0T.png" alt="CFT性能对比" width="1100"/>
</p>

<p align="center"><em>
单样本CFT可持续提升数学与逻辑推理能力。
<strong>左图：</strong>Qwen与LLaMA模型在6个数学推理基准测试上的平均准确率，对比基础模型、监督微调（SFT）、RLVR以及仅使用单个训练样本的CFT模型性能。
<strong>右图：</strong>Qwen2.5-Math-7B模型在3个逻辑推理基准测试（BigBench Extra Hard子任务）上的域内准确率。
在两个领域中，仅使用单个问题的CFT模型均显著优于标准SFT，并以极低的计算量达到甚至超越强化学习的性能。
</em></p>

## 引用

如果您认为我们的工作对您有帮助，请按以下方式引用：

bibtex
@article{wang2025unleashing,
title={Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem},
author={Wang, Yubo and Nie, Ping and Zou, Kai and Wu, Lijun and Chen, Wenhu},
journal={arXiv preprint arXiv:2506.03295},
year={2025}
}