VISLA (Variance and Invariance to Semantic and Lexical Alterations) benchmark

VISLA数据集由达尔豪斯大学和向量研究所创建，旨在评估语言模型对语义和词汇变化的敏感性。数据集包含1613个样本，分为通用和空间两个子集，分别测试模型对词汇变化和空间关系描述的理解能力。数据集通过定义三元组句子与图像的关系，评估模型在多模态和单模态环境下的表现。此外，数据集还特别设计了硬负例，以增加评估的严格性。

The VISLA dataset was developed by Dalhousie University and the Vector Institute, aiming to evaluate the sensitivity of language models to semantic and lexical variations. It contains 1,613 samples, divided into two subsets: general and spatial, which respectively test the model's ability to understand lexical variations and spatial relationship descriptions. By defining the relationship between triplet sentences and images, the dataset evaluates the model's performance across both multimodal and unimodal environments. Furthermore, the dataset specially designs hard negatives to increase the rigor of the evaluation.