Open-AgentRL-30K

<div align="center">

<h1>Demystifying Reinforcement Learning in Agentic Reasoning<h1>

<p align="center"> <a href="https://arxiv.org/abs/2510.11701"> <img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv&logoColor=red" alt="Paper on arXiv"/> </a> <a href="https://github.com/Gen-Verse/Open-AgentRL"> <img src="https://img.shields.io/badge/Open--AgentRL-GitHub-black?logo=github&logoColor=white" alt="Open-AgentRL on GitHub"/> </a> <a href="https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-30K"> <img src="https://img.shields.io/badge/30K_RL_Dataset-Hugging%20Face-orange?logo=huggingface&logoColor=yellow" alt="30K RL Dataset"/> </a> <a href="https://huggingface.co/Gen-Verse/DemyAgent-4B"> <img src="https://img.shields.io/badge/DemyAgent--4B-Hugging%20Face-FFCC00?logo=huggingface&logoColor=yellow" alt="DemyAgent-4B Model"/> </a> </p> </div>

## 🎯 About This Repository
This repository contains the **30K Agentic RL Data**, which consists of 17k [DAPO-Math](https://huggingface.co/datasets/BytedTsinghua-SIA/DAPO-Math-17k) dataset, 3K verifiable science data from [MegaScience](https://huggingface.co/datasets/MegaScience/MegaScience) and our self-curated LeetCode data along with data from [Skywork-OR1](https://huggingface.co/datasets/Skywork/Skywork-OR1-RL-Data).
## 🌟 Introduction
In our work, we systematically investigate three dimensions of agentic RL: **data, algorithms, and reasoning modes**. Our findings reveal
- 🎯 **Data Quality Matters**: Real end-to-end trajectories and high-diversity datasets significantly outperform synthetic alternatives
- ⚡ **Training Efficiency**: Exploration-friendly techniques like reward clipping and entropy maintenance boost training efficiency
- 🧠 **Reasoning Strategy**: Deliberative reasoning with selective tool calls surpasses frequent invocation or verbose self-reasoning
We contribute high-quality SFT and RL datasets, demonstrating that **simple recipes enable even 4B models to outperform 32B models** on the most challenging reasoning benchmarks.
## 📦 Resources
| **Type** | **Name** | **Link** |
| --------- | ------------------- | ------------------------------------------------------------ |
| 📊 Dataset | 3K Agentic SFT Data | [🤗 HuggingFace](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-SFT-3K) |
| 📊 Dataset | **30K Agentic RL Data** | [🤗 HuggingFace](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-30K) |
| 🤖 Model | Qwen2.5-7B-RA-SFT | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/Qwen2.5-7B-RA-SFT) |
| 🤖 Model | Qwen3-4B-RA-SFT | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/Qwen3-4B-RA-SFT) |
| 🤖 Model | DemyAgent-4B | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/DemyAgent-4B) |
## 📝 Citation
```bibtex
@article{yu2025demystify,
title={Demystifying Reinforcement Learning in Agentic Reasoning},
author={Yu, Zhaochen and Yang, Ling and Zou, Jiaru and Yan, Shuicheng and Wang, Mengdi},
journal={arXiv preprint arXiv:2510.11701},
year={2025}
}
```

<div align="center">

<h1>揭开智能体推理（Agentic Reasoning）中强化学习（Reinforcement Learning）的神秘面纱</h1>

<p align="center"> <a href="https://arxiv.org/abs/2510.11701"> <img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv&logoColor=red" alt="arXiv论文"/> </a> <a href="https://github.com/Gen-Verse/Open-AgentRL"> <img src="https://img.shields.io/badge/Open--AgentRL-GitHub-black?logo=github&logoColor=white" alt="Open-AgentRL源码仓库"/> </a> <a href="https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-30K"> <img src="https://img.shields.io/badge/30K_RL_Dataset-Hugging%20Face-orange?logo=huggingface&logoColor=yellow" alt="30K强化学习数据集"/> </a> <a href="https://huggingface.co/Gen-Verse/DemyAgent-4B"> <img src="https://img.shields.io/badge/DemyAgent--4B-Hugging%20Face-FFCC00?logo=huggingface&logoColor=yellow" alt="DemyAgent-4B模型"/> </a> </p> </div>

## 🎯 本仓库说明
本仓库收录**30K智能体强化学习数据集（30K Agentic RL Data）**，其构成包含17k条[DAPO-Math](https://huggingface.co/datasets/BytedTsinghua-SIA/DAPO-Math-17k)数据集、来自[MegaScience](https://huggingface.co/datasets/MegaScience/MegaScience)的3k条可验证科学数据，以及我们自主整理的LeetCode数据集与[Skywork-OR1](https://huggingface.co/datasets/Skywork/Skywork-OR1-RL-Data)提供的数据集。

## 🌟 研究概述
本研究从**数据、算法与推理模式**三个维度对智能体强化学习展开系统性探究，核心结论如下：
- 🎯 **数据质量至关重要**：真实的端到端轨迹与高多样性数据集的表现显著优于合成数据集
- ⚡ **训练效率优化**：奖励裁剪、熵维护等适配探索的技术可有效提升训练效率
- 🧠 **推理策略优选**：结合选择性工具调用的审慎推理，相较于频繁调用工具或冗长的自我推理，表现更为出色
本研究贡献了高质量的监督微调（Supervised Fine-Tuning, SFT）与强化学习数据集，证明**简单的训练配方可使4B参数模型在极具挑战性的推理基准测试中超越32B参数模型**。

## 📦 资源列表
| **类型** | **名称** | **链接** |
| --------- | ------------------- | ------------------------------------------------------------ |
| 📊 数据集 | 3K智能体监督微调数据 | [🤗 Hugging Face](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-SFT-3K) |
| 📊 数据集 | **30K智能体强化学习数据集** | [🤗 Hugging Face](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-30K) |
| 🤖 模型 | Qwen2.5-7B-RA-SFT | [🤗 Hugging Face](https://huggingface.co/Gen-Verse/Qwen2.5-7B-RA-SFT) |
| 🤖 模型 | Qwen3-4B-RA-SFT | [🤗 Hugging Face](https://huggingface.co/Gen-Verse/Qwen3-4B-RA-SFT) |
| 🤖 模型 | DemyAgent-4B | [🤗 Hugging Face](https://huggingface.co/Gen-Verse/DemyAgent-4B) |

## 📝 引用格式
bibtex
@article{yu2025demystify,
title={Demystifying Reinforcement Learning in Agentic Reasoning},
author={Yu, Zhaochen and Yang, Ling and Zou, Jiaru and Yan, Shuicheng and Wang, Mengdi},
journal={arXiv preprint arXiv:2510.11701},
year={2025}
}