ViP-Bench

该数据集名为ViP-Bench，包含303组精心设计的图像与问题配对，旨在全面评估视觉提示跟随能力。这些配对分为六个类别：物体识别、光学字符识别、知识、数学、物体关系推理和语言生成。在默认的划分中，视觉提示由紧致的边界框组成。该数据集的规模为303组配对，其任务是评估视觉提示跟随能力。This dataset, named ViP-Bench, contains 303 meticulously designed image-question pairs, aiming to comprehensively evaluate visual prompt following capabilities. These pairs are categorized into six categories: object recognition, optical character recognition (OCR), knowledge, mathematics, object relational reasoning, and language generation. In the default setup, visual prompts are composed of compact bounding boxes. This dataset has a total of 303 image-question pairs, and its task is to evaluate visual prompt following abilities.

搜集汇总

背景与挑战

背景概述

ViP-Bench是第一个专为零样本区域级理解设计的基准数据集，用于全面评估多模态模型在视觉提示理解方面的能力。它包含303个样本，覆盖识别、OCR、知识、数学、对象关系推理和语言生成六种能力，并支持边界框和人类标注的任意视觉提示两种格式。该数据集采用CC BY NC 4.0许可，仅限于非商业研究用途。