vlms-are-biased

# Vision Language Models are Biased <a href="https://github.com/anvo25/vlms-are-biased/blob/main/static/icons/assumption.png"></a>

<div align="center">
<p style="font-size: 20px;">by
<a href="https://anvo25.github.io/">An Vo</a>^1*,
<a href="https://nkn002.github.io/">Khai-Nguyen Nguyen</a>^2*,
<a href="https://taesiri.ai/">Mohammad Reza Taesiri</a>³, <br>
<a href="https://www.linkedin.com/in/dang-thi-tuong-vy-00a357278/">Vy Tuong Dang</a>¹,
<a href="https://anhnguyen.me/research/">Anh Totti Nguyen</a>^4†,
<a href="https://www.resl.kaist.ac.kr/members/director">Daeyoung Kim</a>^1†
</p>
<p>
^*Equal contribution    ^†Equal advising<br>
¹KAIST, ²College of William and Mary, ³University of Alberta, ⁴Auburn University
</p>
</div>

<div align="center">

[](https://vlmsarebiased.github.io)
[](https://arxiv.org/abs/2505.23941) <!-- Placeholder link -->
[](https://github.com/anvo25/vlms-are-biased)
[](https://huggingface.co/datasets/anvo25/vlms-are-biased)

</div>

---

**TLDR:** State-of-the-art Vision Language Models (VLMs) perform perfectly on counting tasks with original images but fail catastrophically (e.g., 100% → 17.05% accuracy) when familiar objects are subtly modified. This reveals a strong reliance on memorized knowledge over genuine visual analysis. We introduce VLMBias, a benchmark designed to expose this critical flaw.

## Abstract
*Large language models (LLMs) memorize a vast amount of prior knowledge from the Internet that help them on downstream tasks but also may notoriously sway their outputs towards wrong or biased answers. In this work, we test how the knowledge about popular subjects hurt the accuracy of vision language models (VLMs) on standard, objective visual tasks of counting and identification. We find that state-of-the-art VLMs are strongly biased (e.g, unable to recognize a fourth stripe has been added to a 3-stripe Adidas logo) scoring an average of 17.05% accuracy in counting (e.g., counting stripes in an Adidas-like logo) across 7 diverse domains from animals, logos, chess, boardgames, optical illusions, to patterned grids. Insert text (e.g., “Adidas”) describing the subject name into the counterfactual image further decreases VLM accuracy. The biases in VLMs are so strong that instructing them to double-check their results or rely exclusively on image details to answer improves counting accuracy by only +2 points, on average. Our work presents an interesting failure mode in VLMs and an automated framework for testing VLM biases. Code and data are available at: [vlmsarebiased.github.io](https://vlmsarebiased.github.io).*

## Dataset Overview

The VLMBias dataset comprises image-question pairs across **7 diverse domains**: Animals, Logos, National Flags, Chess Pieces, Board Games, Optical Illusions, and Patterned Grids. For each domain, we provide counterfactual images with subtle modifications designed to test genuine visual counting and reasoning against memorized biases. The dataset includes tasks such as counting object parts (e.g., legs, stripes, stars, pieces, grid lines) and identifying anomalies or changes.

# 视觉语言模型存在偏见 <a href="https://github.com/anvo25/vlms-are-biased/blob/main/static/icons/assumption.png"></a>

<div align="center">
<p style="font-size: 20px;">by
<a href="https://anvo25.github.io/">An Vo</a>^1*,
<a href="https://nkn002.github.io/">Khai-Nguyen Nguyen</a>^2*,
<a href="https://taesiri.ai/">Mohammad Reza Taesiri</a>³, <br>
<a href="https://www.linkedin.com/in/dang-thi-tuong-vy-00a357278/">Vy Tuong Dang</a>¹,
<a href="https://anhnguyen.me/research/">Anh Totti Nguyen</a>^4†,
<a href="https://www.resl.kaist.ac.kr/members/director">Daeyoung Kim</a>^1†
</p>
<p>
^*同等贡献    ^†同等指导<br>
¹韩国科学技术院（KAIST）, ²威廉与玛丽学院, ³阿尔伯塔大学, ⁴奥本大学
</p>
</div>

<div align="center">

[](https://vlmsarebiased.github.io)
[](https://arxiv.org/abs/2505.23941)
[](https://github.com/anvo25/vlms-are-biased)
[](https://huggingface.co/datasets/anvo25/vlms-are-biased)

</div>

---

**TLDR：** 当前顶尖的视觉语言模型（Vision Language Models, VLMs）在原始图像的计数任务上表现完美，但当熟悉的物体被细微修改后，其性能会灾难性下降（例如准确率从100%降至17.05%）。这表明VLMs严重依赖记忆中的知识，而非真正的视觉分析。我们提出VLMBias基准测试集，用于揭示这一关键缺陷。

## 摘要
大语言模型（Large Language Models, LLMs）会从互联网中记忆海量先验知识，这些知识虽能辅助其完成下游任务，但也可能会显著地将其输出引导至错误或存在偏见的结果。在本研究中，我们测试了针对常见物体的先验知识如何损害视觉语言模型（VLMs）在标准客观的视觉计数与识别任务中的准确率。我们发现，当前顶尖的VLMs存在极强的偏见：例如，当阿迪达斯（Adidas）的3条纹标志被添加第四条条纹时，模型无法识别这一变化；在涵盖动物、商标、国家国旗、国际象棋、桌面游戏、视错觉、图案网格共7个多样化领域的计数任务中，模型的平均计数准确率仅为17.05%。向经过修改的反事实图像中输入描述物体名称的文本（例如"Adidas"），会进一步降低VLMs的准确率。VLMs的偏见极强，即便指示模型反复检查结果或仅依赖图像细节作答，其计数准确率平均仅提升2个百分点。本研究揭示了VLMs中一种有趣的失效模式，并提供了一个用于测试VLM偏见的自动化框架。相关代码与数据集可通过以下链接获取：[vlmsarebiased.github.io](https://vlmsarebiased.github.io)。

## 数据集概览
VLMBias数据集包含覆盖7个多样化领域的图像-问题对：动物、商标、国家国旗、国际象棋棋子、桌面游戏、视错觉以及图案网格。针对每个领域，我们均提供了经过细微修改的反事实图像，用于测试模型是否基于真实视觉计数与推理能力作答，而非依赖记忆中的偏见。该数据集包含的任务类型包括：计数物体部件（例如肢体、条纹、星标、棋子、网格线）以及识别异常或修改之处。