MMFineReason-SFT-123K

<div align="center"> <h1>MMFineReason-SFT-123K</h1> <p><strong>The Hardest 7% — Less Data, More Reasoning</strong></p> </div> <div align="center"> [](https://arxiv.org/abs/2601.21821) [](https://mmfinereason.github.io/) [](https://huggingface.co/collections/OpenDataArena/mmfinereason) </div> --- ## 📖 Overview **MMFineReason-SFT-123K** is a difficulty-filtered subset of [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M), containing only the **hardest 7%** of samples where Qwen3-VL-4B-Thinking consistently fails (pass rate = 0). ### 🎯 Key Highlights - **123K Challenging Samples**: Only instances where a 4B thinking model fails all 4 inference attempts - **Efficient Training**: Comparable performance to full 1.8M dataset with only 7% of the data - **High-Quality CoT**: Same long-form reasoning annotations from Qwen3-VL-235B-A22B-Thinking --- ## 🔍 Difficulty-Based Filtering <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/pass_rate_distribution.png" width="100%" alt="Pass Rate Distribution"> <figcaption><em>Pass rate distribution across sub-datasets. MMFineReason-SFT-123K contains only samples with pass rate = 0.</em></figcaption> </figure> We use **Qwen3-VL-4B-Thinking** as a difficulty proxy: - Generate 4 independent responses per sample - Compute pass rate (0.0 to 1.0) based on correctness - **Pass rate = 0**: Sample is challenging even for thinking models → included in this subset | Subset | Filter Criteria | Samples | % of Full | |--------|-----------------|---------|-----------| | MMFineReason-1.8M | All filtered data | 1,770,926 | 100% | | MMFineReason-SFT-586K | Pass rate ≠ 1 | 586,000 | 33% | | **MMFineReason-SFT-123K** | **Pass rate = 0** | **123,000** | **7%** | --- ## 📊 "Less is More" Results <figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/ablation_data_efficiency.png" width="100%" alt="Data Efficiency Analysis"> <figcaption><em>Performance comparison across different data scales. Training on 7% hardest samples achieves comparable results to full dataset.</em></figcaption> </figure> **Key Findings:** - Removing 67% easy samples (pass rate = 1) **improves** performance by 0.6 points - Training on only 7% hardest samples still **surpasses** Qwen3-VL-8B-Thinking - Challenging samples provide the majority of training signal; easy samples add noise rather than value --- ## 🔧 Data Schema Same schema as [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M): | Field | Description | |-------|-------------| | `source` | Origin dataset name (e.g., "Geometry3K", "MMR1", "BMMR") | | `id` | Unique sample identifier within the source dataset | | `original_question` | Raw question text as obtained from the source | | `original_answer` | Raw answer as obtained from the source | | `image` | Visual input (PIL Image) | | `question` | Cleaned, standardized question in English | | `answer` | Verified answer extracted and standardized | | `qwen3vl_235b_instruct_caption` | Dense visual description generated by Qwen3-VL-235B-A22B-Instruct | | `qwen3vl_235b_thinking_response` | Long-form Chain-of-Thought reasoning generated by Qwen3-VL-235B-A22B-Thinking | | `qwen3vl_4b_pass_rate` | Difficulty proxy based on Qwen3-VL-4B-Thinking's performance (0.0 = hardest, 1.0 = easiest) | | `is_consistent` | Boolean indicating whether generated reasoning matches ground truth | | `consistency_analysis` | Detailed analysis of consistency verification | --- ## 📚 Citation ```bibtex @misc{lin2026mmfinereasonclosingmultimodalreasoning, title={MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods}, author={Honglin Lin and Zheng Liu and Yun Zhu and Chonghan Qin and Juekai Lin and Xiaoran Shang and Conghui He and Wentao Zhang and Lijun Wu}, year={2026}, eprint={2601.21821}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2601.21821}, } ``` --- ## 📄 License [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0). See [MMFineReason-1.8M](https://huggingface.co/datasets/OpenDataArena/MMFineReason-1.8M) for full acknowledgments.