Vision-CAIR/InfiniBench

InfiniBench是一个用于评估大型多模态模型在超长视频理解任务中的综合基准。该数据集包含超长视频（平均时长76.34分钟）、大量的问题-答案对（108.2K）、多样化的题目类型（包括多选题和开放性问题）以及人类中心化的视频来源（如电影和日常电视节目）。数据集的设计旨在测试模型在九种不同技能上的表现，并包含需要批判性思维和全面理解的‘电影剧透问题’。通过该基准，作者评估了现有的多模态模型，发现即使是表现最好的模型（如Gemini）在该基准上的表现也面临显著挑战。

InfiniBench is a comprehensive benchmark for evaluating large multimodal models in very long video understanding. The dataset includes long videos averaging 76.34 minutes, with 108.2K question-answer pairs. It covers nine different skills and includes both multiple-choice and open-ended questions. The videos are sourced from movies and daily TV shows, focusing on human-centric questions like Movie Spoiler Questions. The dataset aims to stimulate research in long video and human-level understanding.