timeblindness/spooky-bench

SpookyBench是一个专为评估视频语言模型（VLMs）理解纯粹时间模式的能力而设计的新颖基准数据集。该数据集包含四个类别的451个视频：文本、物体图像、动态场景和形状。每个视频在单独的帧中表现为随机噪声，但作为时间序列查看时则显示出有意义的内容（单词、物体等）。这种设计揭示了当前VLMs的一个关键局限性，即它们通常严重依赖空间信息，并且在从纯粹时间序列中提取意义时存在困难。

SpookyBench is a novel benchmark dataset designed to evaluate the ability of video-language models (VLMs) to understand purely temporal patterns, independent of spatial cues. The dataset consists of 451 videos across four categories: Text, Object Images, Dynamic Scenes, and Shapes. Each video appears as random noise in individual frames, but reveals meaningful content (words, objects, etc.) when viewed as a temporal sequence. This design exposes a critical limitation in current VLMs, which often heavily rely on spatial information and struggle to extract meaning from purely temporal sequences.