HiTZ/MATE

MATE数据集是一个包含5500个问题回答示例的基准，用于评估视觉语言模型(VLMs)在视觉和文本模态间实体链接的能力。每个场景由3到10个具有不同颜色、形状、材料和大小的新型3D几何对象组成，并分别以视觉模态（图像）和文本模态（JSON格式的对象及其属性列表）表示。MATE数据集基于CLEVR数据集进行扩展，增加了独特的对象名称和额外的形状。每个示例包含一个问题，问题具有指示属性和目标属性。在cross_modal配置中，指示和目标属性位于不同的模态中，而在unimodal配置中，两者位于相同模态中。

The MATE dataset consists of 5,500 question-answering examples designed to evaluate the cross-modal entity linking capabilities of vision-language models (VLMs). Each scene is composed of three to ten novel 3D geometric objects with varying colors, shapes, materials, and sizes, represented in both visual (image) and textual (JSON-formatted list of objects and their attributes) modalities. The MATE dataset extends the CLEVR dataset with unique object names and additional shapes. Each example includes one question with a pointer attribute and a target attribute. In the cross_modal configuration, the pointer and target attributes are in different modalities, while in the unimodal configuration, both attributes are in the same modality.