FlowCE Evaluation Dataset

该数据集是对多模态大型语言模型（MLLMs）在流程图相关任务上进行全面评估的集合，涵盖了推理、定位识别、信息提取、逻辑验证和总结等多个维度。此外，该数据集还提供了不同模型在不同任务上的详细性能指标，突显了开源和闭源MLLMs的优势与不足。所涉及的模型规模多样，包括参数量从34亿到超过130亿的私有和开源模型。任务重点在于评估MLLMs在流程图理解方面的表现。

This dataset is a comprehensive benchmark for evaluating multimodal large language models (MLLMs) on flowchart-related tasks, covering multiple dimensions including reasoning, localization and recognition, information extraction, logical verification, and summarization. Additionally, this dataset provides detailed performance metrics of various models across different tasks, highlighting the strengths and limitations of both open-source and closed-source MLLMs. The models involved have diverse scales, with parameter counts ranging from 3.4 billion to over 13 billion for both private and open-source models. The tasks primarily focus on evaluating the performance of MLLMs in flowchart understanding.