OpenDataArena/ODA-Fin-RL-12k

---
license: apache-2.0
language:
- en
- zh
task_categories:
- text-generation
- question-answering
tags:
- finance
- reasoning
- reinforcement-learning
- GRPO
- domain-adaptation
size_categories:
- 10K<n<100K
---

<div align="center">
<h1>Unlocking Data Value in Finance: A Study on Distillation
and Difficulty-Aware Training</h1>

</div>

<div align="center">

[](https://arxiv.org/abs/2603.07223)
[](https://huggingface.co/collections/OpenDataArena/oda-finance)

</div>

---

## 📖 Overview

**ODA-Fin-RL-12K** is a carefully curated dataset for reinforcement learning (RL) in financial domain, comprising **12,187 hard-but-verifiable samples**. Designed to complement ODA-Fin-SFT-318K, this dataset targets challenging financial reasoning tasks with concise, reliably verifiable answers—optimized for RL training.

---

## 🎯 Key Highlights

- **12K Hard Samples**: Curated based on >50% failure rate from ODA-Fin-SFT-8B
- **Verifiable Rewards**: Final answers limited to ≤16 tokens for reliable automated verification
- **Reward Precision-Diversity Trade-off**: 22 unique data sources with maximum final answer length of 16 token
- **Domain Coverage**: Financial QA, Sentiment Analysis, Numerical Reasoning

---

## 🏗️ Data Construction Pipeline

<figure align="center">
<img src="imgs/data_construction_pipeline.png" width="100%" alt="Pass Rate Distribution">
<figcaption><em>Data construction pipeline of ODA-Fin-SFT-318k and ODA-Fin-RL-12k.</em></figcaption>
</figure>

### Stage 1: Difficulty-Based Filtering
1. **Base Model Evaluation**: Use ODA-Fin-SFT-8B to evaluate all 318K SFT samples
2. **Multiple Rollouts**: Generate 4 answers per sample with temperature=0.6
3. **Failure Rate Calculation**:
```
Failure Rate = (Number of incorrect answers) / 4
```
4. **Hard Sample Selection**: Retain samples with failure rate > 50%

### Stage 2: Verifiability Filtering
1. **Ground Truth Analysis**: Extract final answers from gold labels
2. **Token Length Constraint**: Keep only samples where `len(answer_tokens) ≤ 16`

### Stage 3: Diversity Enhancement
- **Source Distribution**: Maintain representation from 22 distinct datasets
- **Task Balance**: Ensure coverage across financial QA, sentiment, and reasoning

### Final Output
- **12,187 samples** optimized for RL training
- **0% trivial samples** (pass rate = 1.0)
- **100% verifiable** via automated methods

---

## 🔧 Data Schema — ODA-Fin-RL-12k

**File format**: Parquet (`train.parquet` / `test.parquet`)
**Splits**: `train` (12,187 samples) · `test` (200 samples)

| Field | Sub-field | Type | Description |
|---|---|---|---|
| `data_source` | — | `string` | Origin dataset name on HuggingFace (e.g., `"Josephgflowers/Finance-Instruct-500k"`, `"FinGPT/fingpt-sentiment-train"`). 22 unique sources in total |
| `prompt` | — | `list[dict]` | Chat-format input as a single-turn conversation list. Always contains one element: `[{"role": "user", "content": "<question>"}]` |
| `ability` | — | `string` | Domain tag. Uniformly `"finance"` across all samples |
| `reward_model` | `ground_truth` | `string` | The short-form expected answer used as the RL reward signal (e.g., `"negative"`, `"neutral"`, `"He was shocked by her sudden announcement."`). Token length is always ≤ 16 |
| | `style` | `string` | Verification method. Always `"model"` — reward is computed by a model-based verifier (CompassVerifier-7B for classification/numerical tasks; Qwen3-235B-A22B-Instruct for analysis tasks) |
| `extra_info` | `id` | `string` | MD5 hash unique identifier for the sample |
| | `index` | `int` | Sequential index within the dataset |
| | `question` | `string` | Raw question text (identical to `prompt[0]["content"]`) |
| | `answer` | `string` | Ground-truth answer (identical to `reward_model["ground_truth"]`) |
| | `task` | `string` | Task category. One of: `"Financial QA"`, `"Sentiment Analysis"`, `"Financial Numerical Analysis"`, `"Information Extraction"`, `"Financial Analysis"`, `"Other"` |
| | `split` | `string` | Dataset split: `"train"` or `"test"` |
| | `token_length` | `int` | Token count of the ground-truth answer. Range: 0–15 (enforced ≤ 16 for verifiability) |

---

### Example Record (train)

```json
{
"data_source": "takala/financial_phrasebank",
"prompt": [
{
"role": "user",
"content": "Analyze the sentiment of the following financial statement and classify it as positive, negative, or neutral.\nLook out for vintage fabric cushion covers, '70s coffee pots, ceramic serving dishes, cocktail glasses, and stainless steel party dishes."
}
],
"ability": "finance",
"reward_model": {
"ground_truth": "neutral",
"style": "model"
},
"extra_info": {
"id": "b09cb83081f9ec0a110ea893afca7930",
"index": 1,
"question": "Analyze the sentiment of the following financial statement and classify it as positive, negative, or neutral.\nLook out for vintage fabric cushion covers, '70s coffee pots, ceramic serving dishes, cocktail glasses, and stainless steel party dishes.",
"answer": "neutral",
"task": "Sentiment Analysis",
"split": "train",
"token_length": 1
}
}
```
---

## 📊 Dataset Statistics

<figure align="center">
<img src="imgs/rl_data_distribution.png" width="100%" alt="Pass Rate Distribution">
<figcaption><em>Data source and task distribution.</em></figcaption>
</figure>

- Total samples: 12,187
- Domains: General Finance, Sentiment Analysis, Numerical Reasoning, Forecasting, Risk Analysis

---

## 📈 Benchmark Results

### RL vs SFT Performance

<figure align="center">
<img src="imgs/main_results_table.png" width="100%" alt="Pass Rate Distribution">
<figcaption><em>Main Results: ODA-Fin-RL achieves top three performance across most benchmarks. 'FinIQ', 'HL' and 'CFQA' refer to FinanceIQ, Headlines, and ConvFinQA benchmarks.</em></figcaption>
</figure>

- The comparison between ODA-Fin-SFT-8B and ODA-FinRL-8B highlights the critical role of the RL stage.
- The RL process yields consistent improvements across almost all metrics, raising the average score from 72.1% to 74.6%.

### Ablation: Verifier Design & Answer Length

<figure align="center">
<img src="imgs/ablation_study.png" width="100%" alt="Pass Rate Distribution">
<figcaption><em>Ablation study on RL stage. ‘#Tok’ denotes the maximum final answer’s token length constraint used for filtering RL training data, ‘#Size’ refers the data size. ‘FinIQ’, ‘HL’ and ‘CFQA’ refer to FinanceIQ, Headlines, and ConvFinQA benchmarks.</em></figcaption>
</figure>

- **Optimal constraint**: 16 tokens balances verifier precision and task diversity
- Overly strict (1 token) limits coverage; too loose (128 tokens) introduces noisy rewards
- Model-based verification (CompassVerifier-7B) outperforms pure rule matching

---

## 🔬 Analysis & Insights

### 1. Hard-but-Verifiable Principle

The dataset construction follows the principle that **RL gains come from hard-but-verifiable samples**:
- **Hard** (pass_rate ≤ 0.5): Forces the model to learn non-trivial reasoning
- **Verifiable** (≤16 tokens): Ensures reliable reward signal for policy updates

### 2. Reward Precision-Diversity Trade-off

<figure align="center">
<img src="imgs/ablation_fig.png" width="100%" alt="Pass Rate Distribution">
<figcaption><em>Ablation study on RL stage. ‘#Tok’ denotes the maximum final answer’s token length constraint used for filtering RL training data, ‘#Size’ refers the data size. ‘FinIQ’, ‘HL’ and ‘CFQA’ refer to FinanceIQ, Headlines, and ConvFinQA benchmarks.</em></figcaption>
</figure>

**Key Insight**: The 16-token constraint hits the "sweet spot" where verifier can still judge reliably while allowing diverse task types.

### 3. Impact of Base Model Quality

**Key Insight**: Strong SFT initialization is critical—RL from weak base models can degrade performance.

---

## 📚 Citation

```bibtex
@misc{cao2026unlockingdatavaluefinance,
title={Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training},
author={Chuxue Cao and Honglin Lin and Zhanping Zhong and Xin Gao and Mengzhang Cai and Conghui He and Sirui Han and Lijun Wu},
year={2026},
eprint={2603.07223},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2603.07223},
}

```

---
## 📄 License

This model is released under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0). The training data (ODA-Fin-SFT-318K) aggregates from 25+ open-source repositories, each with their own licenses.

---

## 🤝 Acknowledgments

We thank the creators of DianJin-R1-Data, Agentar-DeepFinance-100K, financial_phrasebank, Finance-Instruct-500k, and others. We also thank the Qwen team for the powerful Qwen3 series models.

---

## 🔗 Related Resources

- **SFT Dataset**: [ODA-Fin-SFT-318K](https://huggingface.co/datasets/OpenDataArena/ODA-Fin-SFT-318k)
<!-- - **RL Dataset**: [ODA-Fin-RL-12K](https://huggingface.co/datasets/OpenDataArena/ODA-Fin-RL-12K) -->
- **SFT Model**: [ODA-Fin-SFT-8B](https://huggingface.co/OpenDataArena/ODA-Fin-SFT-8B)
- **RL Model**: [ODA-Fin-RL-8B](https://huggingface.co/OpenDataArena/ODA-Fin-RL-8B)

<!-- - **Paper**: [arXiv:2512.XXXXX](https://arxiv.org/abs/2512.XXXXX) -->

---
许可证：Apache 2.0
语言：
- 英语
- 中文
任务类别：
- 文本生成
- 问答
标签：
- 金融
- 推理
- 强化学习（Reinforcement Learning, RL）
- GRPO
- 领域自适应
数据规模：
- 10K<n<100K
---

<div align="center">
<h1>解锁金融数据价值：蒸馏与难度感知训练研究</h1>
</div>

<div align="center">

[](https://arxiv.org/abs/2603.07223)
[](https://huggingface.co/collections/OpenDataArena/oda-finance)

</div>

---

## 📖 数据集概览

**ODA-Fin-RL-12K** 是一款专为金融领域强化学习（Reinforcement Learning, RL）设计的精选数据集，包含**12,187个兼具难度与可验证性的样本**。本数据集旨在作为ODA-Fin-SFT-318K的补充，聚焦于拥有简洁且可靠可验证答案的高难度金融推理任务，专为RL训练优化。

---

## 🎯 核心亮点

- **12K高难度样本**：基于ODA-Fin-SFT-8B的失败率>50%的样本精选而来
- **可验证奖励**：最终答案限制在≤16个Token，支持可靠的自动化验证
- **奖励精度-多样性权衡**：包含22个独特数据源，最终答案最大长度为16个Token
- **领域覆盖**：金融问答、情感分析、数值推理

---

## 🏗️ 数据构建流程

<figure align="center">
<img src="imgs/data_construction_pipeline.png" width="100%" alt="通过率分布">
<figcaption><em>ODA-Fin-SFT-318K与ODA-Fin-RL-12K的数据构建流程。</em></figcaption>
</figure>

### 阶段1：基于难度的筛选
1. **基础模型评估**：使用ODA-Fin-SFT-8B评估全部318K个监督微调（Supervised Fine-Tuning, SFT）样本
2. **多次推理生成**：对每个样本以温度系数0.6生成4个答案
3. **失败率计算**：

失败率 = 错误答案数量 / 4

4. **高难度样本筛选**：保留失败率>50%的样本

### 阶段2：基于可验证性的筛选
1. **标准答案分析**：从金标准标签中提取最终答案
2. **Token长度约束**：仅保留`len(answer_tokens) ≤16`的样本

### 阶段3：多样性增强
- **源数据分布**：保留22个不同数据集的样本分布
- **任务均衡**：确保覆盖金融问答、情感分析与推理任务

### 最终输出
- **12,187个专为RL训练优化的样本**
- **无 trivial 样本**（通过率=1.0）
- **100%可通过自动化方法验证**

---

## 🔧 数据结构 — ODA-Fin-RL-12K

**文件格式**：Parquet格式（`train.parquet` / `test.parquet`）
**数据集划分**：训练集（12,187个样本）· 测试集（200个样本）

| 字段 | 子字段 | 类型 | 描述 |
|---|---|---|---|
| `data_source` | — | `string` | HuggingFace上的源数据集名称（例如：`"Josephgflowers/Finance-Instruct-500k"`、`"FinGPT/fingpt-sentiment-train"`）。总计22个独特数据源 |
| `prompt` | — | `list[dict]` | 单轮对话格式的聊天式输入，始终仅包含一个元素：`[{"role": "user", "content": "<question>"}]` |
| `ability` | — | `string` | 领域标签，所有样本统一为`"finance"` |
| `reward_model` | `ground_truth` | `string` | 用作RL奖励信号的短格式预期答案（例如：`"negative"`、`"neutral"`、`"He was shocked by her sudden announcement."`）。Token长度始终≤16 |
| | `style` | `string` | 验证方法，始终为`"model"` — 奖励由基于模型的验证器计算（分类/数值任务使用CompassVerifier-7B；分析任务使用Qwen3-235B-A22B-Instruct） |
| `extra_info` | `id` | `string` | 样本的唯一MD5哈希标识符 |
| | `index` | `int` | 数据集中的顺序索引 |
| | `question` | `string` | 原始问题文本（与`prompt[0]["content"]`完全一致） |
| | `answer` | `string` | 标准答案（与`reward_model["ground_truth"]`完全一致） |
| | `task` | `string` | 任务类别，可选值为：`"Financial QA"`、`"Sentiment Analysis"`、`"Financial Numerical Analysis"`、`"Information Extraction"`、`"Financial Analysis"`、`"Other"` |
| | `split` | `string` | 数据集划分：`"train"`或`"test"` |
| | `token_length` | `int` | 标准答案的Token数量，范围为0–15（为保证可验证性强制≤16） |

---

### 训练集示例样本

json
{
"data_source": "takala/financial_phrasebank",
"prompt": [
{
"role": "user",
"content": "请分析以下金融声明的情感倾向，并将其分类为积极、消极或中性。
请注意复古布艺靠垫套、70年代咖啡壶、陶瓷餐具、鸡尾酒酒杯和不锈钢派对餐具。"
}
],
"ability": "finance",
"reward_model": {
"ground_truth": "neutral",
"style": "model"
},
"extra_info": {
"id": "b09cb83081f9ec0a110ea893afca7930",
"index": 1,
"question": "请分析以下金融声明的情感倾向，并将其分类为积极、消极或中性。
请注意复古布艺靠垫套、70年代咖啡壶、陶瓷餐具、鸡尾酒酒杯和不锈钢派对餐具。",
"answer": "neutral",
"task": "Sentiment Analysis",
"split": "train",
"token_length": 1
}
}

---

## 📊 数据集统计

<figure align="center">
<img src="imgs/rl_data_distribution.png" width="100%" alt="数据分布">
<figcaption><em>数据源与任务分布。</em></figcaption>
</figure>

- 总样本数：12,187
- 覆盖领域：通用金融、情感分析、数值推理、预测、风险分析

---

## 📈 基准测试结果

### 强化学习与监督微调性能对比

<figure align="center">
<img src="imgs/main_results_table.png" width="100%" alt="基准测试结果">
<figcaption><em>主要结果：ODA-Fin-RL在多数基准测试中位列前三。'FinIQ'、'HL'与'CFQA'分别指代FinanceIQ、Headlines与ConvFinQA基准数据集。</em></figcaption>
</figure>

- ODA-Fin-SFT-8B与ODA-FinRL-8B的对比凸显了RL阶段的关键作用。
- RL过程在几乎所有指标上均带来持续提升，平均得分从72.1%提升至74.6%。

### 消融实验：验证器设计与答案长度

<figure align="center">
<img src="imgs/ablation_study.png" width="100%" alt="消融实验结果">
<figcaption><em>RL阶段的消融实验。‘#Tok’表示用于筛选RL训练数据的最终答案最大Token长度约束，‘#Size’指代数据集规模。‘FinIQ’、‘HL’与‘CFQA’分别指代FinanceIQ、Headlines与ConvFinQA基准数据集。</em></figcaption>
</figure>

- **最优约束**：16个Token的长度限制在验证器精度与任务多样性之间取得了平衡
- 过严的约束（1个Token）会限制覆盖范围；过松的约束（128个Token）则会引入带有噪声的奖励信号
- 基于模型的验证（CompassVerifier-7B）优于纯规则匹配

---

## 🔬 分析与洞察

### 1. 高难度且可验证原则

本数据集的构建遵循**RL的收益来源于高难度且可验证样本**的原则：
- **高难度**（通过率≤0.5）：迫使模型学习非 trivial 的推理逻辑
- **可验证**（≤16个Token）：为策略更新提供可靠的奖励信号

### 2. 奖励精度与多样性的权衡

<figure align="center">
<img src="imgs/ablation_fig.png" width="100%" alt="消融实验结果">
<figcaption><em>RL阶段的消融实验。‘#Tok’表示用于筛选RL训练数据的最终答案最大Token长度约束，‘#Size’指代数据集规模。‘FinIQ’、‘HL’与‘CFQA’分别指代FinanceIQ、Headlines与ConvFinQA基准数据集。</em></figcaption>
</figure>

**核心洞察**：16个Token的约束达到了“甜蜜点”，既能让验证器可靠判断，又允许多样化的任务类型。

### 3. 基础模型质量的影响

**核心洞察**：强大的SFT初始化至关重要——从弱基础模型开始的RL训练可能会降低性能。

---

## 📚 引用

bibtex
@misc{cao2026unlockingdatavaluefinance,
title={Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training},
author={Chuxue Cao and Honglin Lin and Zhanping Zhong and Xin Gao and Mengzhang Cai and Conghui He and Sirui Han and Lijun Wu},
year={2026},
eprint={2603.07223},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2603.07223},
}

---
## 📄 许可证

本数据集采用[Apache 2.0许可证](https://opensource.org/licenses/Apache-2.0)发布。训练数据集（ODA-Fin-SFT-318K）整合了25个以上的开源仓库，每个仓库均有其独立许可证。

---

## 🤝 致谢

我们感谢DianJin-R1-Data、Agentar-DeepFinance-100K、financial_phrasebank、Finance-Instruct-500k等数据集的创作者。同时感谢Qwen团队推出的高性能Qwen3系列模型。

---

## 🔗 相关资源

- **SFT数据集**：[ODA-Fin-SFT-318K](https://huggingface.co/datasets/OpenDataArena/ODA-Fin-SFT-318k)
- **SFT模型**：[ODA-Fin-SFT-8B](https://huggingface.co/OpenDataArena/ODA-Fin-SFT-8B)
- **RL模型**：[ODA-Fin-RL-8B](https://huggingface.co/OpenDataArena/ODA-Fin-RL-8B)