RVLab-CBNU/SALUS

SALUS是一个多选视觉问答（VQA）数据集，旨在公平和稳健地评估视觉语言模型（VLMs）。与广泛使用的基于COCO数据集的VQA基准不同，SALUS是基于Docci数据集构建的，后者包含高度详细的人工编写的图像描述。通过GPT-4o的帮助生成10选1的多选VQA对，并经过严格的人工验证以确保质量和正确性。数据集还强调了减少预训练重叠和提高评估公平性的特点，以及通过结构化设计实现不确定性感知评估。数据集包含36,241个问答对（训练集23,905个，测试集12,336个），每个问题有10个选项，每个图像对应1到3个问答对。

SALUS is a multiple-choice Visual Question Answering (VQA) dataset designed to enable fair and robust evaluation of vision-language models (VLMs). Unlike widely used VQA benchmarks that rely on the COCO dataset, SALUS is constructed from the Docci Dataset, a dataset consisting of highly detailed, human-written image descriptions. Using these descriptions, we generate 10-choice multiple-choice VQA pairs with the assistance of GPT-4o, followed by strict human verification to ensure quality and correctness. The dataset emphasizes reducing pretraining overlap and improving evaluation fairness, while also enabling uncertainty-aware evaluation through its structured design. It contains 36,241 QA pairs (Train: 23,905, Test: 12,336), with 10 choices per question and 1~3 QA pairs per image.