MMFineReason

<div align="center"> <h1>MMFineReason-Full-2.3M</h1> <p><strong>The Complete Pre-Selection Dataset — Before Quality Filtering</strong></p> </div> <div align="center"> [](https://arxiv.org/abs/2601.xxxxx) [](https://mmfinereason.github.io/) [](https://huggingface.co/collections/OpenDataArena/mmfinereason) </div> --- ## 📖 Overview **MMFineReason-Full-2.3M** is the complete pre-selection dataset containing **2.3M samples** and **8.8B solution tokens**, generated through our reasoning distillation pipeline **before** the data selection stage. This dataset includes all samples that passed basic template and length validation, but have **not** undergone correctness verification filtering. ### 🎯 Key Characteristics - **2,286,130 Total Samples** with **8.8B Solution Tokens** - **Pre-Selection Data**: Contains both consistent and inconsistent samples - **Research Purpose**: Enables study of data quality, filtering strategies, and consistency patterns ### ⚠️ Important Note This dataset is released for **research purposes**. For training, we recommend using the filtered [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M) which removes ~20% inconsistent samples. This full version is valuable for: - Studying consistency patterns across different domains - Developing better filtering strategies - Analyzing the relationship between difficulty and consistency - Ablation studies on data quality --- ## 🔍 Consistency Analysis <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/consistency_analysis.png" width="100%" alt="Consistency Analysis"> <figcaption><em>Consistency analysis across visual instruction tuning datasets. The ratio of samples where predictions from Qwen3-VL-235B-A22B-Thinking align with ground truth ("Consistent") versus cases of disagreement ("Inconsistent").</em></figcaption> </figure> We verify each sample by comparing the extracted answer from the `<answer>` tag against the ground truth. - **High-Consistency Datasets**: ScienceQA (96%), TQA (86%), MMR1 (85%) — well-posed samples with clear ground truths - **Challenging Datasets**: VisualSphinx (36%), Raven (39%), Geometry3K (54%) — abstract reasoning tasks with inherent difficulty or ambiguous samples - **Overall**: ~79% of samples are consistent (1.81M / 2.29M), forming the basis for MMFineReason-1.8M --- ## 📊 Dataset Statistics ### Dataset Composition <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/table_composition.png" width="100%" alt="Dataset Composition"> <figcaption><em>Dataset composition of MMFineReason-Full. Total: 2,286,130 samples, 8,785,423,669 tokens.</em></figcaption> </figure> --- ### Data Cleaning Statistics <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/table_cleaning.png" width="100%" alt="Data Cleaning Statistics"> <figcaption><em>Data Cleaning Statistics. "Filt. (Len)" = length-based filtering, "Filt. (Tem)" = template validation errors.</em></figcaption> </figure> Starting from 2,372,320 raw samples: - **Length Filtering**: Removed 1,806 samples exceeding context limits or anomalously short - **Template Filtering**: Removed 84,484 samples with parsing errors or malformed `<think>/<answer>` structure - **Retention Rate**: 96.36% (2,286,130 samples remain) Most datasets show >95% retention. Slightly lower rates for VisualSphinx (93.1%) and Geometry3K (92.4%) due to complex formatting requirements. --- ### Pass Rate and Consistency Statistics <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/table_consistency.png" width="100%" alt="Pass Rate and Consistency Statistics"> <figcaption><em>Pass Rate (PR) and Consistency Statistics by Dataset.</em></figcaption> </figure> **Key Findings:** - Strong positive correlation between Pass Rate and Consistency Rate - High-consistency datasets (ScienceQA, AI2D, TQA) have clear ground truths and high pass rates (>0.8) - Challenging tasks (Raven, VisualSphinx) show low pass rates (<0.25) and consistency (<40%) --- ## 🔧 Data Schema Same schema as [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M): | Field | Description | |-------|-------------| | `source` | Origin dataset name (e.g., "Geometry3K", "MMR1", "BMMR") | | `id` | Unique sample identifier within the source dataset | | `original_question` | Raw question text as obtained from the source | | `original_answer` | Raw answer as obtained from the source | | `image` | Visual input (PIL Image) | | `question` | Cleaned, standardized question in English | | `answer` | Verified answer extracted and standardized | | `qwen3vl_235b_instruct_caption` | Dense visual description generated by Qwen3-VL-235B-A22B-Instruct | | `qwen3vl_235b_thinking_response` | Long-form Chain-of-Thought reasoning generated by Qwen3-VL-235B-A22B-Thinking | | `qwen3vl_4b_pass_rate` | Difficulty proxy based on Qwen3-VL-4B-Thinking's performance (0.0 = hardest, 1.0 = easiest) | | `is_consistent` | Boolean indicating whether generated reasoning matches ground truth | | `consistency_analysis` | Detailed analysis of consistency verification | --- ## 📊 Comparison with Filtered Versions | Dataset | Samples | Tokens | Consistency | Purpose | |---------|---------|--------|-------------|---------| | **MMFineReason-Full-2.3M** | 2,286,130 | 8.8B | Mixed (79% consistent) | Research & Analysis | | [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M) | 1,770,926 | 5.1B | 100% consistent | Training (SFT) | | [MMFineReason-SFT-123K](https://huggingface.co/datasets/OpenDataArena/MMFineReason-SFT-123K) | 123,000 | ~0.4B | 100% consistent | Efficient Training | --- ## 📚 Citation ```bibtex @misc{lin2026mmfinereasonclosingmultimodalreasoning, title={MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods}, author={Honglin Lin and Zheng Liu and Yun Zhu and Chonghan Qin and Juekai Lin and Xiaoran Shang and Conghui He and Wentao Zhang and Lijun Wu}, year={2026}, eprint={2601.21821}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2601.21821}, } ``` --- ## 📄 License [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0). See [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M) for full acknowledgments.