"DRef: A Benchmark with Diverse Referring Expressions for Object Comprehension of Vision-Language Models"

"Our benchmark, Diverse Referring Expressions for Object Comprehension (DRef),comprises 30,566 diverse referring expressions across 2,290 images.The dataset aims to evaluate the robustness and consistency of grounding models under real-world scenarios.Each target object includes an average of 8.3 distinct positive expressions and 5.0 negative expressions denoting non-existent objects. The negative referring expressions are designed to fool a model by modifying the modifiers of existing texts. For each referring expression, we also provide at least one description perspective, including 2D position, 3D position, relative position, possible usage, possible status, interaction, attribute, size, reasoning, etc. For the dataset structure, please refer to the raw_dataset_structure.md"