OpenMMReasoner/OpenMMReasoner-SFT-874K

---
task_categories:
- image-text-to-text
configs:
- config_name: llava_cot
data_files:
- split: train
path: parquet/llava_cot.parquet
- config_name: OpenVLThinker-sft-iter3
data_files:
- split: train
path: parquet/OpenVLThinker-sft-iter3.parquet
- config_name: WeMath
data_files:
- split: train
path: parquet/WeMath.parquet
- config_name: m1_sft
data_files:
- split: train
path: parquet/m1_sft.parquet
- config_name: mmr1
data_files:
- split: train
path: parquet/mmr1_filtered.parquet
---

# OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe

<div align="center">

[](https://huggingface.co/collections/lmms-lab/openmmreasoner)
[](https://arxiv.org/abs/2511.16334)
[](https://evolvinglmms-lab.github.io/OpenMMReasoner/)
[](https://github.com/EvolvingLMMs-Lab/OpenMMReasoner)
</div>

## Introduction

OpenMMReasoner is a dataset introduced in the paper [OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe](https://arxiv.org/abs/2511.16334). It supports the development of multimodal reasoning capabilities, utilizing a fully transparent two-stage recipe spanning supervised fine-tuning (SFT) and reinforcement learning (RL). The dataset includes an 874K-sample cold-start dataset for foundational reasoning and a 74K-sample dataset across diverse domains to further sharpen these abilities.

## Data Preparation and Usage

This repository contains the ColdStart Data used to train **[OpenMMReasoner-ColdStart](https://huggingface.co/OpenMMReasoner/OpenMMReasoner-ColdStart)**. We use **[lmms-engine](https://github.com/EvolvingLMMs-Lab/lmms-engine)** as the training framework.

To get started, you can download all required datasets from Hugging Face using the provided script:

```bash
bash examples/openmmreasoner/download_data.sh [LOCAL_DIR]
```
Replace `[LOCAL_DIR]` with your desired local directory (defaults to `./data`).

After downloading, if the dataset includes image archives, unzip them to extract all images into the correct directory:

```bash
unzip "image_zip/*.zip" -d <output_folder>
```

Next, prepare a data YAML file to point the training framework to your dataset:

```yaml
datasets:
- path: /path/to/your/dataset/llava_cot.parquet
data_folder: "/path/to/your/dataset/images"
data_type: parquet
```

This configuration ensures that `lmms-engine` can correctly locate and load the images.

For more detailed instructions and concrete examples on SFT training, RL training, and evaluation, please refer to our [GitHub repository](https://github.com/EvolvingLMMs-Lab/OpenMMReasoner).

## Citation

If you find OpenMMReasoner useful for your research and applications, please cite using this BibTeX:

```bibtex
@misc{zhang2025openmmreasonerpushingfrontiersmultimodal,
title={OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe},
author={Kaichen Zhang and Keming Wu and Zuhao Yang and Bo Li and Kairui Hu and Bin Wang and Ziwei Liu and Xingxuan Li and Lidong Bing},
year={2025},
eprint={2511.16334},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2511.16334},
}
```

task_categories:
- 图像-文本转文本
configs:
- config_name: llava_cot
data_files:
- split: train
path: parquet/llava_cot.parquet
- config_name: OpenVLThinker-sft-iter3
data_files:
- split: train
path: parquet/OpenVLThinker-sft-iter3.parquet
- config_name: WeMath
data_files:
- split: train
path: parquet/WeMath.parquet
- config_name: m1_sft
data_files:
- split: train
path: parquet/m1_sft.parquet
- config_name: mmr1
data_files:
- split: train
path: parquet/mmr1_filtered.parquet

---

# OpenMMReasoner：依托开源通用范式，拓展多模态推理前沿

<div align="center">

[](https://huggingface.co/collections/lmms-lab/openmmreasoner)
[](https://arxiv.org/abs/2511.16334)
[](https://evolvinglmms-lab.github.io/OpenMMReasoner/)
[](https://github.com/EvolvingLMMs-Lab/OpenMMReasoner)
</div>

## 引言
OpenMMReasoner是论文《OpenMMReasoner：依托开源通用范式，拓展多模态推理前沿》（https://arxiv.org/abs/2511.16334）中提出的数据集。该数据集旨在赋能多模态推理能力的研发，采用全透明的两阶段范式，涵盖监督微调（Supervised Fine-Tuning, SFT）与强化学习（Reinforcement Learning, RL）。数据集包含87.4万样本的冷启动基础推理数据集，以及7.4万样本的多领域数据集，用于进一步精进模型的推理能力。

## 数据准备与使用
本仓库包含用于训练**OpenMMReasoner-ColdStart**的冷启动数据集。我们采用**lmms-engine**作为训练框架。
快速上手可通过提供的脚本从Hugging Face下载所有所需数据集：
bash
bash examples/openmmreasoner/download_data.sh [LOCAL_DIR]

将`[LOCAL_DIR]`替换为你期望的本地目录（默认路径为`./data`）。
下载完成后，若数据集包含图像压缩包，请将其解压至指定目录以提取所有图像：
bash
unzip "image_zip/*.zip" -d <output_folder>

随后需准备数据YAML配置文件，以指引训练框架定位数据集：
yaml
datasets:
- path: /path/to/your/dataset/llava_cot.parquet
data_folder: "/path/to/your/dataset/images"
data_type: parquet

该配置可确保`lmms-engine`能够正确定位并加载图像数据。
如需了解监督微调、强化学习训练与评估的详细指南及实操示例，请参阅我们的[GitHub仓库](https://github.com/EvolvingLMMs-Lab/OpenMMReasoner)。

## 引用
若您的研究与应用中使用了OpenMMReasoner，请引用如下BibTeX条目：
bibtex
@misc{zhang2025openmmreasonerpushingfrontiersmultimodal,
title={OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe},
author={Kaichen Zhang and Keming Wu and Zuhao Yang and Bo Li and Kairui Hu and Bin Wang and Ziwei Liu and Xingxuan Li and Lidong Bing},
year={2025},
eprint={2511.16334},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2511.16334},
}