M3STR

该数据集名为M3STR，是一个创新性的基准测试，旨在评估多模态大型语言模型（MLLMs）在理解和推理以视觉形式呈现的结构化知识方面的能力。该数据集不仅促进了主流开源MLLMs在各项任务中的性能评估，还揭示了它们在抽象视觉理解和推理方面的优势和局限性。具体来说，该数据集涉及三项与抽象视觉推理以及多模态地图理解相关的任务。

This dataset, named M3STR, is an innovative benchmark designed to evaluate the capabilities of Multimodal Large Language Models (MLLMs) in understanding and reasoning over visually presented structured knowledge. This benchmark not only facilitates the performance evaluation of mainstream open-source MLLMs across various tasks, but also reveals their strengths and limitations in abstract visual understanding and reasoning. Specifically, this dataset includes three tasks related to abstract visual reasoning and multimodal map understanding.