turing-motors/RIO-Bench

RIO-Bench是一个统一基准测试，旨在评估视觉语言模型（VLMs）在排版攻击鲁棒性和文本识别方面的性能。它通过一个名为RIO-VQA的新任务，要求VLMs自适应地决定何时读取或忽略图像中的文本。数据集包含多个配置，对应不同的子集和任务，每个配置都有train和val两个分割。数据字段因任务类型而异，但通常包括图像、问题、答案和元数据。该数据集基于TextVQA等现有资源构建，并利用Llama-3.1-8B-Instruct等模型生成对抗攻击。

RIO-Bench is a unified benchmark designed to evaluate typographic-attack robustness and text recognition in Vision-Language Models (VLMs). It introduces a novel task called RIO-VQA that requires VLMs to adaptively decide when to read or ignore text in images. The dataset includes multiple configurations corresponding to different subsets and tasks, with each config having both train and val splits. The data fields vary by task type but commonly include images, questions, answers, and metadata. The dataset is built upon existing resources like TextVQA and utilizes models like Llama-3.1-8B-Instruct for generating adversarial attacks.