Logics-MLLM/OmniParsingBench

Name: Logics-MLLM/OmniParsingBench
Creator: Logics-MLLM
Published: 2026-05-10T03:27:01+08:00

Hugging Face2026-04-08 更新2026-05-10 收录

多模态解析评估

MLLM解析基准

数据链接：

https://hf-mirror.com/datasets/Logics-MLLM/OmniParsingBench 数据链接链接失效反馈

官方服务：

资源简介：

--- license: apache-2.0 language: - zh - en pretty_name: OmniParsingBench tags: - parsing - chart - document - multimodal configs: - config_name: default data_files: - split: natural_image path: data/natural_image.jsonl - split: graphics path: data/graphics.jsonl - split: audio path: data/audio.jsonl - split: natural_video path: data/natural_video.jsonl - split: textrich_video path: data/textrich_video.jsonl --- <div align="center"> <img src="logo.png" width="80%"> </div> 🤗 <a href="https://huggingface.co/Logics-MLLM/Logics-Parsing-Omni">Model</a>&nbsp&nbsp | &nbsp&nbsp📑 <a href="https://arxiv.org/pdf/2603.09677">Technical Report</a>&nbsp&nbsp | &nbsp&nbsp💻 <a href="https://github.com/alibaba/Logics-Parsing/tree/main/Logics-Parsing-Omni">GitHub</a> **OmniParsingBench** is a comprehensive, large-scale, and high-quality evaluation corpus designed to rigorously evaluate the unified parsing capabilities of Multimodal Large Language Models (MLLMs) across diverse modalities. Unlike traditional single-task benchmarks, OmniParsingBench assesses the full spectrum of parsing performance—from fundamental signal detection to complex semantic reasoning—across six primary domains: **Document, Natural Image, Graphics, Audio, Natural Video, and Text-Rich Video**. ## 📖 Evaluation Framework & Metrics Our evaluation framework strictly aligns with a proposed three-stage architecture, systematically assessing performance across different cognitive levels: - **L1 - Holistic Detection:** Spatio-temporal grounding and classification. - **L2 - Fine-grained Recognition:** Symbol extraction, attribute identification, and structural recovery. - **L3 - Multi-level Interpreting:** Semantic consistency and hallucination resistance. To provide a concise view of model capabilities, we aggregate these fine-grained metrics into two core scores, alongside an overall metric: * **Perception (Perc.):** Evaluates signal precision and structural fidelity (dominating L1 and L2). * **Cognition (Cog.):** Evaluates logical reasoning and semantic understanding (dominating L3). * **Overall (Ovr.):** The comprehensive performance metric across all levels. ## 🏆 Leaderboard ### Overall Performance <div align="center"> <table> <thead> <tr> <th rowspan="2" align="left" valign="middle">Model</th> <th colspan="3" align="center">Natural Image</th> <th colspan="3" align="center">Graphics</th> <th colspan="1" align="center">Document</th> <th colspan="3" align="center">Audio</th> <th colspan="3" align="center">Natural Video</th> <th colspan="3" align="center">Text-Rich Video</th> </tr> <tr> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Perc.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> </tr> </thead> <tbody> <tr> <td align="left">Gemini-3-Pro</td> <td align="center">61.20</td> <td align="center">55.96</td> <td align="center">66.44</td> <td align="center">87.03</td> <td align="center">84.21</td> <td align="center">87.43</td> <td align="center">87.01</td> <td align="center">79.40</td> <td align="center">72.90</td> <td align="center">85.89</td> <td align="center">63.40</td> <td align="center">57.87</td> <td align="center">68.92</td> <td align="center">64.37</td> <td align="center">58.54</td> <td align="center">70.20</td> </tr> <tr> <td align="left">GPT-5.2</td> <td align="center">39.94</td> <td align="center">37.77</td> <td align="center">42.12</td> <td align="center">82.71</td> <td align="center">69.86</td> <td align="center">91.48</td> <td align="center">77.43</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3.5-397B-A17B</td> <td align="center">57.40</td> <td align="center">56.95</td> <td align="center">57.85</td> <td align="center">82.81</td> <td align="center">73.77</td> <td align="center">83.13</td> <td align="center">81.09</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-VL-235B-A22B</td> <td align="center">58.61</td> <td align="center">56.23</td> <td align="center">60.99</td> <td align="center">79.49</td> <td align="center">71.51</td> <td align="center">83.46</td> <td align="center">84.47</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-VL-30B-A3B</td> <td align="center">50.92</td> <td align="center">48.91</td> <td align="center">52.94</td> <td align="center">73.25</td> <td align="center">65.71</td> <td align="center">79.36</td> <td align="center">78.94</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-Omni-30B-A3B</td> <td align="center">47.36</td> <td align="center">46.85</td> <td align="center">47.88</td> <td align="center">77.46</td> <td align="center">70.75</td> <td align="center">78.25</td> <td align="center">73.50</td> <td align="center">75.17</td> <td align="center">62.13</td> <td align="center">88.22</td> <td align="center">45.23</td> <td align="center">34.15</td> <td align="center">56.32</td> <td align="center">26.86</td> <td align="center">10.22</td> <td align="center">43.50</td> </tr> <tr> <td align="left">Logics-Parsing-Omni (Ours)</td> <td align="center">59.07</td> <td align="center">53.77</td> <td align="center">64.37</td> <td align="center">88.66</td> <td align="center">82.01</td> <td align="center">92.12</td> <td align="center">84.90</td> <td align="center">79.63</td> <td align="center">69.27</td> <td align="center">89.99</td> <td align="center">61.12</td> <td align="center">56.09</td> <td align="center">66.15</td> <td align="center">69.12</td> <td align="center">57.39</td> <td align="center">80.85</td> </tr> </tbody> </table> Note: Bold text indicates the best result, and underlined text indicates the second-best result. </div> ### 📊 Results Analysis As detailed in the table above, **Logics-Parsing-Omni** demonstrates highly competitive or state-of-the-art capabilities across all six diverse modalities: * **Dominance in Complex Modalities:** Our model consistently surpasses all evaluated baselines—including the leading proprietary **Gemini-3-Pro**—in the *Overall* and *Cognition* metrics of the *Graphics, Audio, and Text-Rich Video* domains. * **Exceptional Semantic Understanding:** The superiority is particularly pronounced in the **Cognition** metric, where Logics-Parsing-Omni exhibits exceptional logical reasoning and semantic understanding, achieving top-tier scores such as **92.12** in Graphics and **80.85** in Text-Rich Video. * **Leading Open-Weight Performance:** While Gemini-3-Pro maintains an advantage in the fundamental *Perception* of Natural Images, Graphics, Audio, and Documents, as well as a marginal lead in Natural Video, our model significantly outperforms other open-weight counterparts (e.g., the Qwen series) in nearly all metrics. These quantitative results validate the efficacy of our L1–L3 architecture, demonstrating that Logics-Parsing-Omni successfully bridges fundamental signal detection with complex multi-modal interpreting. ## 📊 Dataset Overview | Split | Modality | Source | Size | |-------|----------|--------|------| | `natural_image` | Image | [Pexels](https://www.pexels.com), [Wikimedia Commons](https://commons.wikimedia.org) | 1,000 | | `graphics` | Image | Synthesized (charts & geometric figures) | 1,000 | | `audio` | Audio | [YouTube](https://www.youtube.com) | 1,014 | | `natural_video` | Video | [YouTube](https://www.youtube.com) | 1,121 | | `textrich_video` | Video | [YouTube](https://www.youtube.com) | 259 | | `document` | Document | [SkylenAge](https://skylenage.alibabagroup.com/sla/evaluation/detail?id=OFW6tlGUt2F4merPuEF26) | 900 pages | | **Total** | | | **5294** | > **Data Fields:** `ID` · `URL` · `Start_time/End_time` · `Cognition` · `Perception` · `Split` ## 📝 Citation If you find OmniParsingBench or our model useful in your research, please consider citing our technical report: ```bibtex @article{logicsparsingomni2026, title={Logics-Parsing-Omni: Bridging Fine-Grained Perception and Semantic Cognition in Multimodal Parsing}, author={Logics Team}, journal={arXiv preprint arXiv:2603.09677}, year={2026} }

许可证：Apache-2.0 语言： - 中文 - 英文友好名称：OmniParsingBench 标签： - 解析（parsing） - 图表（chart） - 文档（document） - 多模态（multimodal）配置项： - 配置名称：default 数据文件： - 拆分：natural_image（自然图像）路径：data/natural_image.jsonl - 拆分：graphics（图表）路径：data/graphics.jsonl - 拆分：audio（音频）路径：data/audio.jsonl - 拆分：natural_video（自然视频）路径：data/natural_video.jsonl - 拆分：textrich_video（文本丰富视频）路径：data/textrich_video.jsonl <div align="center"> <img src="logo.png" width="80%"> </div> 🤗 <a href="https://huggingface.co/Logics-MLLM/Logics-Parsing-Omni">模型</a>&nbsp&nbsp | &nbsp&nbsp📑 <a href="https://arxiv.org/pdf/2603.09677">技术报告</a>&nbsp&nbsp | &nbsp&nbsp💻 <a href="https://github.com/alibaba/Logics-Parsing/tree/main/Logics-Parsing-Omni">GitHub</a> **OmniParsingBench**是一个全面、大规模且高质量的评测语料库，旨在严格评估多模态大语言模型（Multimodal Large Language Model, MLLMs）在多样化模态下的统一解析能力。与传统单任务评测基准不同，OmniParsingBench覆盖六大核心领域的全维度解析性能——从基础信号检测到复杂语义推理：**文档、自然图像、图表、音频、自然视频与文本丰富视频**。 ## 📖 评估框架与指标我们的评估框架严格遵循提出的三阶段架构，系统评估不同认知层级的性能： - **L1 - 全局检测（Holistic Detection）：** 时空定位与分类 - **L2 - 细粒度识别（Fine-grained Recognition）：** 符号提取、属性识别与结构恢复 - **L3 - 多级解读（Multi-level Interpreting）：** 语义一致性与抗幻觉能力为简洁展示模型能力，我们将这些细粒度指标整合为两个核心分数与一项综合指标： * **感知能力（Perception, Perc.）：** 评估信号精度与结构保真度，主导L1与L2层级 * **认知能力（Cognition, Cog.）：** 评估逻辑推理与语义理解能力，主导L3层级 * **综合性能（Overall, Ovr.）：** 覆盖所有层级的综合性能指标 ## 🏆 排行榜 ### 综合性能 <div align="center"> <table> <thead> <tr> <th rowspan="2" align="left" valign="middle">模型</th> <th colspan="3" align="center">自然图像</th> <th colspan="3" align="center">图表</th> <th colspan="1" align="center">文档</th> <th colspan="3" align="center">音频</th> <th colspan="3" align="center">自然视频</th> <th colspan="3" align="center">文本丰富视频</th> </tr> <tr> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Perc.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> <th align="center">Ovr.</th> <th align="center">Perc.</th> <th align="center">Cog.</th> </tr> </thead> <tbody> <tr> <td align="left">Gemini-3-Pro</td> <td align="center">61.20</td> <td align="center">55.96</td> <td align="center">66.44</td> <td align="center">87.03</td> <td align="center">84.21</td> <td align="center">87.43</td> <td align="center">87.01</td> <td align="center">79.40</td> <td align="center">72.90</td> <td align="center">85.89</td> <td align="center">63.40</td> <td align="center">57.87</td> <td align="center">68.92</td> <td align="center">64.37</td> <td align="center">58.54</td> <td align="center">70.20</td> </tr> <tr> <td align="left">GPT-5.2</td> <td align="center">39.94</td> <td align="center">37.77</td> <td align="center">42.12</td> <td align="center">82.71</td> <td align="center">69.86</td> <td align="center">91.48</td> <td align="center">77.43</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3.5-397B-A17B</td> <td align="center">57.40</td> <td align="center">56.95</td> <td align="center">57.85</td> <td align="center">82.81</td> <td align="center">73.77</td> <td align="center">83.13</td> <td align="center">81.09</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-VL-235B-A22B</td> <td align="center">58.61</td> <td align="center">56.23</td> <td align="center">60.99</td> <td align="center">79.49</td> <td align="center">71.51</td> <td align="center">83.46</td> <td align="center">84.47</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-VL-30B-A3B</td> <td align="center">50.92</td> <td align="center">48.91</td> <td align="center">52.94</td> <td align="center">73.25</td> <td align="center">65.71</td> <td align="center">79.36</td> <td align="center">78.94</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> <td align="center">--</td> </tr> <tr> <td align="left">Qwen3-Omni-30B-A3B</td> <td align="center">47.36</td> <td align="center">46.85</td> <td align="center">47.88</td> <td align="center">77.46</td> <td align="center">70.75</td> <td align="center">78.25</td> <td align="center">73.50</td> <td align="center">75.17</td> <td align="center">62.13</td> <td align="center">88.22</td> <td align="center">45.23</td> <td align="center">34.15</td> <td align="center">56.32</td> <td align="center">26.86</td> <td align="center">10.22</td> <td align="center">43.50</td> </tr> <tr> <td align="left">Logics-Parsing-Omni（本文提出模型）</td> <td align="center">59.07</td> <td align="center">53.77</td> <td align="center">64.37</td> <td align="center">88.66</td> <td align="center">82.01</td> <td align="center">92.12</td> <td align="center">84.90</td> <td align="center">79.63</td> <td align="center">69.27</td> <td align="center">89.99</td> <td align="center">61.12</td> <td align="center">56.09</td> <td align="center">66.15</td> <td align="center">69.12</td> <td align="center">57.39</td> <td align="center">80.85</td> </tr> </tbody> </table> 注：粗体文本表示最优结果，下划线文本表示次优结果。 </div> ### 📊 结果分析如上述表格所示，**Logics-Parsing-Omni**在全部六大模态中展现出极具竞争力甚至顶尖的性能： * **复杂模态优势：** 本模型在*图表、音频与文本丰富视频*领域的*综合性能*与*认知能力*指标上，持续超越所有参评基线——包括领先的闭源模型**Gemini-3-Pro**。 * **卓越语义理解：** 该优势在**认知能力**指标上尤为显著，Logics-Parsing-Omni展现出顶尖的逻辑推理与语义理解能力，在图表领域取得92.12分、文本丰富视频领域取得80.85分的顶级成绩。 * **领先的开源模型性能：** 尽管Gemini-3-Pro在自然图像、图表、音频与文档的基础*感知能力*上保持优势，在自然视频领域也小幅领先，但本模型在几乎所有指标上均显著优于其他开源竞品（例如Qwen系列模型）。这些量化结果验证了我们L1-L3架构的有效性，证明Logics-Parsing-Omni成功实现了基础信号检测与复杂多模态解读之间的衔接。 ## 📊 数据集概览 | 拆分 | 模态 | 来源 | 规模 | |-------|----------|--------|------| | `natural_image`（自然图像） | 图像 | [Pexels](https://www.pexels.com)、[维基共享资源](https://commons.wikimedia.org) | 1000条 | | `graphics`（图表） | 图像 | 合成（图表与几何图形） | 1000条 | | `audio`（音频） | 音频 | [YouTube](https://www.youtube.com) | 1014条 | | `natural_video`（自然视频） | 视频 | [YouTube](https://www.youtube.com) | 1121条 | | `textrich_video`（文本丰富视频） | 视频 | [YouTube](https://www.youtube.com) | 259条 | | `document`（文档） | 文档 | [SkylenAge](https://skylenage.alibabagroup.com/sla/evaluation/detail?id=OFW6tlGUt2F4merPuEF26) | 900页 | | **总计** | | | **5294** | > **数据字段：** `ID`（编号）、`URL`（链接）、`Start_time/End_time`（开始时间/结束时间）、`Cognition`（认知能力）、`Perception`（感知能力）、`Split`（拆分） ## 📝 引用如果您在研究中使用OmniParsingBench或本模型，请引用我们的技术报告： bibtex @article{logicsparsingomni2026, title={Logics-Parsing-Omni: Bridging Fine-Grained Perception and Semantic Cognition in Multimodal Parsing}, author={Logics Team}, journal={arXiv preprint arXiv:2603.09677}, year={2026} }

提供机构：

Logics-MLLM

Logics-MLLM/OmniParsingBench

资源简介：

相关数据集