M3GIA

该数据集是一个基于认知驱动的多语言和多模态基准，旨在评估大规模多语言模型（MLLMs）的一般智力能力，其理论基础是卡特尔-霍恩-卡罗尔（Cattell-Horn-Carroll）的智力模型。该数据集揭示了MLLMs与人类智力在各个认知领域之间的性能差异，并为理解MLLMs的认知结构提供了洞见。该数据集涉及480名参与者，他们回答了1,800个问题，这些问题被分为六个子问卷，按语言分类。任务旨在通过认知评估，对多语言和多模态大型语言模型（MLLMs）的一般智力能力进行评价。

This dataset is a cognition-driven multilingual and multimodal benchmark developed to assess the general intelligence capabilities of large multilingual models (MLLMs), with its theoretical foundation rooted in the Cattell-Horn-Carroll (CHC) model of intelligence. This benchmark reveals the performance gaps between MLLMs and human intelligence across diverse cognitive domains, and provides critical insights into deciphering the cognitive architecture of MLLMs. It encompasses 480 participants who responded to 1,800 questions, which are grouped into six language-categorized sub-questionnaires. The tasks herein aim to evaluate the general intelligence capabilities of multilingual and multimodal large language models (MLLMs) via standardized cognitive assessments.