VRBench

VRBench是一个用于评估大型模型多步推理能力的长叙事视频基准数据集。它包含1010个手动筛选的叙事视频，覆盖8种语言和7个视频类别，适合推理时间关系。我们还提供高质量的分步推理标注，由人类专家进行标注和审查。每个视频包含8-10个复杂的问答对、多步推理链和细粒度的时间戳。为了充分评估模型在多步推理方面的能力，我们提出了一个多阶段评估流程，从过程和结果两个层面评估模型结果。VRBench是第一个既支持多步标注又支持评估的视频推理基准。

VRBench is a long-form narrative video benchmark dataset for evaluating the multi-step reasoning capabilities of large-scale models. It comprises 1010 manually curated narrative videos, covering 8 languages and 7 video categories, and is designed for reasoning about temporal relationships. We also provide high-quality step-by-step reasoning annotations, which are labeled and reviewed by human experts. Each video contains 8 to 10 complex question-answer pairs, multi-step reasoning chains, and fine-grained timestamps. To fully evaluate the multi-step reasoning capabilities of models, we propose a multi-stage evaluation pipeline that assesses model outputs from both procedural and outcome perspectives. VRBench is the first video reasoning benchmark that supports both multi-step annotation and model evaluation.