tsinghua-ee/AVUTBenchmark

音频中心视频理解基准（AVUT）数据集旨在评估多模态大型语言模型（LLM）对视频的理解能力，特别关注音频信息。该数据集通过使用答案排列过滤机制来解决其他基准中存在的文本捷径问题。数据集包括两个主要的注释文件：一个由人工标注者精心创建，另一个由Gemini模型自动生成。

The Audio-centric Video Understanding Benchmark (AVUT) dataset is designed to evaluate the video comprehension capabilities of multimodal Large Language Models (LLMs) with a focus on auditory information. This dataset addresses the text shortcut problem present in other benchmarks by employing an answer permutation-based filtering mechanism. The dataset structure consists of two primary annotation files: one created by human annotators and another generated automatically by the Gemini model.