meta-ai-for-media-research/movie_gen_video_bench

---
language:
- en
pretty_name: Movie Gen Video Benchmark
dataset_info:
features:
- name: prompt
dtype: string
- name: video
dtype: binary
splits:
- name: test_with_generations
num_bytes: 16029316444
num_examples: 1003
- name: test
num_bytes: 113706
num_examples: 1003
download_size: 16029724908
dataset_size: 16029430150
configs:
- config_name: default
data_files:
- split: test_with_generations
path: data/test_with_generations-*
- split: test
path: data/test-*
---

# Dataset Card for the Movie Gen Benchmark
[Movie Gen](https://ai.meta.com/research/movie-gen/) is a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio.
Here, we introduce our evaluation benchmark "Movie Gen Bench Video Bench", as detailed in the [Movie Gen technical report](https://ai.meta.com/static-resource/movie-gen-research-paper) (Section 3.5.2).

To enable fair and easy comparison to Movie Gen for future works on these evaluation benchmarks, we additionally release the non cherry-picked generated videos from Movie Gen on Movie Gen Video Bench.

## Dataset Summary
Movie Gen Video Bench consists of 1003 prompts that cover all the different testing aspects/concepts:

1. human activity (limb and mouth motion, emotions, etc.)
2. animals
3. nature and scenery
4. physics (fluid dynamics, gravity, acceleration, collisions, explosions, etc.)
5. unusual subjects and unusual activities.
Besides a comprehensive coverage of different key testing aspects, the prompts also have a good coverage of high/medium/low motion levels at the same time.



## Dataset Splits
We are releasing two versions of the benchmark:
1. Test (test): This version includes only the prompts, making it easier to download and use the benchmark.
2. Test with Generations (test_with_generations): This version includes both the prompts and the Movie Gen model’s outputs, allowing for comparative evaluation against the Movie Gen model.

## Usage

```python
from datasets import load_dataset

# to download only the prompts
dataset = load_dataset("meta-ai-for-media-research/movie_gen_video_bench_no_generations")["test"]
for example in dataset:
print(example)
break

# to download the prompts and movie gen generations
dataset = load_dataset("meta-ai-for-media-research/movie_gen_video_bench", split="test_with_generations", streaming=True)
for example in dataset:
break

# to display Movie Gen generated video and the prompt on jupyter notebook
import mediapy

with open("tmp.mp4", "wb") as f:
f.write(example["video"])

video = mediapy.read_video("tmp.mp4")
print(example["prompt"])
mediapy.show_video(video)
```

## Licensing Information
Licensed with [CC-BY-NC](https://github.com/facebookresearch/MovieGenBench/blob/main/LICENSE) License.

---
language:
- 英文
pretty_name: Movie Gen Video Benchmark
dataset_info:
特征:
- 名称: 提示词（prompt）
数据类型: string
- 名称: 视频
数据类型: binary
拆分:
- 名称: 测试集含生成结果（test_with_generations）
字节数: 16029316444
样本数: 1003
- 名称: 测试集（test）
字节数: 113706
样本数: 1003
下载大小: 16029724908
数据集大小: 16029430150
配置:
- 配置名称: default
数据文件:
- 拆分: 测试集含生成结果（test_with_generations）
路径: data/test_with_generations-*
- 拆分: 测试集（test）
路径: data/test-*
---

# Movie Gen基准数据集卡片

Movie Gen是一系列基础模型（foundation models），可生成高质量1080p高清视频，支持不同宽高比（aspect ratio）及同步音频（synchronized audio）。在此，我们介绍评估基准（benchmark）——Movie Gen视频基准（Movie Gen Video Bench），详情见《Movie Gen技术报告》（https://ai.meta.com/static-resource/movie-gen-research-paper，第3.5.2节）。

为便于未来相关评估基准（benchmark）研究中与Movie Gen进行公平且便捷的对比，我们额外发布了Movie Gen在Movie Gen视频基准（Movie Gen Video Bench）上生成的非精选视频。

## 数据集摘要
Movie Gen视频基准（Movie Gen Video Bench）包含1003个提示词（prompt），覆盖各类测试维度/概念：
1. 人类活动（肢体与嘴部动作、情绪等）
2. 动物
3. 自然与风景
4. 物理现象（流体动力学（fluid dynamics）、重力、加速度、碰撞、爆炸等）
5. 特殊主体与特殊活动

除全面覆盖不同关键测试维度外，这些提示词还同时涵盖高/中/低运动水平。



## 数据集拆分
我们发布了该基准的两个版本：
1. 测试集（test）：仅包含提示词，便于下载和使用基准。
2. 测试集含生成结果（test_with_generations）：包含提示词与Movie Gen模型的输出结果，可用于与Movie Gen模型进行对比评估。

## 使用方法

python
from datasets import load_dataset

# 仅下载提示词
dataset = load_dataset("meta-ai-for-media-research/movie_gen_video_bench_no_generations")["test"]
for example in dataset:
print(example)
break

# 下载提示词与Movie Gen生成结果
dataset = load_dataset("meta-ai-for-media-research/movie_gen_video_bench", split="test_with_generations", streaming=True)
for example in dataset:
break

# 在Jupyter Notebook中显示Movie Gen生成的视频与提示词
import mediapy

with open("tmp.mp4", "wb") as f:
f.write(example["video"])

video = mediapy.read_video("tmp.mp4")
print(example["prompt"])
mediapy.show_video(video)

## 许可信息
采用[CC-BY-NC](https://github.com/facebookresearch/MovieGenBench/blob/main/LICENSE)许可协议授权。