visual-riddles/visual_riddles

Visual Riddles数据集是一个旨在评估视觉推理和常识理解能力的基准测试。它包含一系列视觉丰富的谜题，每个谜题都配有专门生成的合成图像。这些谜题结合了微妙的视觉线索和日常场景，挑战人类和模型在解释和常识推理方面的能力。数据集包括开放式视觉问答（VQA）、多项选择VQA和自动评分开放式回答等任务。实验结果显示，人类和最先进的视觉与语言模型之间存在显著的性能差距，突出了将常识推理和世界知识整合到模型架构中的挑战。此外，尝试使用文本到图像模型重现基准图像也揭示了视觉谜题带来的独特挑战。

The Visual Riddles dataset is a benchmark designed to assess visual reasoning and commonsense understanding. It comprises a collection of visually rich riddles, each accompanied by a synthetic image generated specifically for the challenge. These riddles are carefully crafted to integrate subtle visual cues with everyday scenarios, challenging both humans and models in their interpretative and commonsense reasoning abilities. The dataset encompasses various tasks, including Open-ended Visual Question Answering (VQA), Multiple-Choice VQA, and Auto-Rating of Open-ended Responses.