AVA-Bench: Atomic Visual Abilities for Vision Foundation Models

As vision foundation models (VFMs) grow in scale and capability, understanding their strengths and weaknesses becomes increasingly important. AVA-Bench is the first benchmark designed to provide fine-grained, transparent evaluation of VFMs by disentangling performance across 14 Atomic Visual Abilities (AVAs)—such as object counting, depth estimation, spatial reasoning, and localization. Unlike conventional Visual Question Answering (VQA) benchmarks, which often conflate multiple visual skills and rely on large language models (LLMs) as heads, AVA-Bench isolates individual visual competencies, ensuring that evaluation reflects the core capabilities of the vision model itself. Each AVA task in the benchmark is constructed with aligned training and test distributions to reduce confounds and sharpen interpretability. The result is a diagnostic suite that generates distinct ability fingerprints for any VFM, allowing researchers and practitioners to identify exactly where a model excels or fails.