metalex001/metalex

MetaLex是一个用于测试视觉语言模型中字面到比喻基础能力的基准数据集。每个项目将一个词汇化隐喻与字面文本解释、隐喻成分的字面图像以及多项选择比喻意义配对。数据集支持多种诊断条件，包括仅隐喻字符串、仅字面文本、仅字面图像、标题中介以及联合文本和图像等，旨在作为多模态比喻语言基础和模态差距分析的诊断基准。

MetaLex is a benchmark for testing literal-to-figurative grounding in vision-language models. Each item pairs a lexicalised metaphor with a literal textual paraphrase, a literal image of the metaphors constituents, and multiple-choice figurative meanings. The benchmark supports several diagnostic conditions, including metaphor string only, literal text only, literal image only, caption-mediated, and joint text and image, intended as a diagnostic benchmark for multimodal figurative-language grounding and modality-gap analysis.