marvex/ShotBench

ShotBench是一个官方测试集，包含3,572个问答对，每个样本都与一张图像或一个视频片段配对。总计包括3,049张图像和464个视频，主要来自获得奥斯卡最佳摄影奖提名的电影，确保了高视觉质量和强烈的电影风格。该数据集旨在评估视觉语言模型（VLMs）对电影语言的理解能力，涵盖了八个不同的摄影维度，为电影中细粒度视觉理解提供了严格的新标准。

ShotBench is the official test set comprising 3,572 question-answer pairs, each paired with either an image or a video clip. In total, it includes 3,049 images and 464 videos, primarily sourced from films that received Oscar nominations for Best Cinematography, ensuring high visual quality and strong cinematic style. The dataset is designed to evaluate Vision-Language Models (VLMs) understanding of cinematic language, covering eight distinct cinematography dimensions, and provides a rigorous new standard for assessing fine-grained visual comprehension in film.