Event-Bench

# Towards Event-oriented Long Video Understanding

<font size=3><div align='center'> [[📖 arXiv Paper]()]</div>

---

## 👀 Overview

We introduce **Event-Bench**, an event-oriented long video understanding benchmark built on existing datasets and human annotations. **Event-Bench** consists of three event understanding abilities and six event-related tasks, including 2,190 test instances to comprehensively evaluate the ability to understand video events.
<p align="center">
<img src="./asset/fig_benchmark.jpg" width="100%" height="100%">
</p>

**Event-Bench** provides a systematic comparison across different kinds of capabilities for existing video MLLMs, and points out the major shortcomings of open-source MLLMs.

## 🔍 Dataset
Download the raw videos in Event-Bench from the [google drive link](https://drive.google.com/file/d/1wjjH2dK-KpaObFdS1yc-TBUTCvXsaLwc/view?usp=sharing).

**License**:
```
Event-Bench is only used for academic research. Commercial use in any form is prohibited.
```

## 🔮 Evaluation Pipeline
Please refer to https://github.com/RUCAIBox/Event-Bench

## 📈 Experimental Results
- **Evaluation results of different Video MLLMs.**

<p align="center">
<img src="./asset/performance.png" width="96%" height="50%">
</p>

## Citation

If you find our work helpful for your research, please consider citing our work.

```bibtex
@misc{du2024eventoriented,
title={Towards Event-oriented Long Video Understanding},
author={Yifan Du and Kun Zhou and Yuqi Huo and Yifan Li and Wayne Xin Zhao and Haoyu Lu and Zijia Zhao and Bingning Wang and Weipeng Chen and Ji-Rong Wen},
year={2024},
eprint={2406.14129},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```

# 面向事件的长视频理解（Towards Event-oriented Long Video Understanding）

<div align='center'> [[📖 arXiv 论文]()]</div>

---

## 👀 概览
我们推出了**事件基准数据集（Event-Bench）**，这是一个基于现有数据集与人工标注构建的面向事件的长视频理解基准测试集。事件基准数据集（Event-Bench）包含三类事件理解能力与六项事件相关任务，共计2190个测试样本，可全面评估视频事件理解能力。
<p align="center">
<img src="./asset/fig_benchmark.jpg" width="100%" height="100%">
</p>

事件基准数据集（Event-Bench）可对现有视频多模态大语言模型（Multimodal Large Language Model, MLLM）的各类能力开展系统性对比分析，并指明开源多模态大语言模型存在的主要短板。

## 🔍 数据集
可从以下谷歌网盘链接下载事件基准数据集（Event-Bench）的原始视频：[谷歌网盘链接](https://drive.google.com/file/d/1wjjH2dK-KpaObFdS1yc-TBUTCvXsaLwc/view?usp=sharing)。

**使用许可**：

Event-Bench 仅可用于学术研究，禁止任何形式的商业使用。

## 🔮 评估流程
请参考 https://github.com/RUCAIBox/Event-Bench

## 📈 实验结果
- **不同视频多模态大语言模型的评估结果**

<p align="center">
<img src="./asset/performance.png" width="96%" height="50%">
</p>

## 引用
若您的研究工作受益于本项目，请引用如下文献：

bibtex
@misc{du2024eventoriented,
title={Towards Event-oriented Long Video Understanding},
author={Yifan Du and Kun Zhou and Yuqi Huo and Yifan Li and Wayne Xin Zhao and Haoyu Lu and Zijia Zhao and Bingning Wang and Weipeng Chen and Ji-Rong Wen},
year={2024},
eprint={2406.14129},
archivePrefix={arXiv},
primaryClass={cs.CV}
}