VLM-CapCurriculum-Perception-Data
收藏资源简介：
VLM-CapCurriculum-Perception (D_perc) 是一个用于视觉语言模型（VLM）分阶段后训练中第一阶段（视觉感知）的数据集，源自ICML 2026论文《从看见到思考：解耦感知与推理改进视觉语言模型的后训练》。该数据集的核心是专门构造的四选一多选题，其问题可以通过细粒度的图像描述来回答，但仅凭图像观察的强视觉语言模型却会答错。通过这种构造方式，数据集旨在隔离模型的感知失败，而非推理失败，从而专门用于提升模型的底层视觉感知能力。每个数据样本包含一个图像、对应的问题、答案、以及基于基础模型Qwen3-VL-8B-Instruct的16次推理结果计算出的预测、正确性和关键的通过率（pass_rate）。通过率作为样本的难度信号，使得数据集可以按照难度（升序为困难优先，降序为简单优先）进行排序，从而支持基于能力与难度的课程学习实验。数据集包含3,360个训练样本，图像源自DOCCI数据集（下采样2倍），共计约14,847个图像文件。该数据集主要用于视觉语言模型在视觉问答和多选任务上的感知能力后训练与评估，特别是与姊妹数据集（TextReasoning, VisualReasoning）结合，构成完整的感知-推理解耦训练流程。
VLM-CapCurriculum-Perception (D_perc) is a dataset for the first stage (visual perception) in phased post-training of visual language models (VLMs), originating from the ICML 2026 paper From Seeing to Thinking: Decoupling Perception and Reasoning to Improve Post-Training of Visual Language Models. The core of this dataset is specially constructed multiple-choice questions with four options, where the questions can be answered through fine-grained image descriptions, but strong visual language models relying solely on image observation will answer incorrectly. This construction isolates perceptual failures rather than reasoning failures, specifically aimed at enhancing the models underlying visual perception capabilities. Each data sample includes an image, corresponding question, answer, and predictions, correctness, and key pass_rate calculated from 16 inference results based on the base model Qwen3-VL-8B-Instruct. The pass_rate serves as a difficulty signal, allowing the dataset to be sorted by difficulty (ascending for difficulty-first, descending for easy-first), thereby supporting curriculum learning experiments based on ability and difficulty. The dataset contains 3,360 training samples, with images sourced from the DOCCI dataset (downsampled by 2x), totaling approximately 14,847 image files. It is primarily used for post-training and evaluation of visual language models in visual question answering and multiple-choice tasks, particularly when combined with sister datasets (TextReasoning, VisualReasoning) to form a complete perception-reasoning decoupled training pipeline.
