MV-Video
收藏资源简介：
MV-Video数据集是一个大规模的多视角视频数据集，由53K个动画3D对象渲染而成。该数据集用于训练Animate3D模型，该模型能够通过多视角视频扩散技术动画化任何3D模型。每个对象渲染16个均匀分布的视角，视角的仰角在0-30度之间随机采样，起始方位角有±11.25度的随机偏移。提供的视频长度为2秒（24帧每秒），对于2-4秒的动画，渲染前2秒；对于超过4秒的动画，渲染前2秒和后2秒。对于有超过6个动画的对象，随机采样6个动画以避免过拟合。数据集包含多个压缩文件，解压后包含视频文件和元信息文件。数据集的部分数据被过滤，提供的数量略少于论文中报告的数量。约7.7K个对象被标记为最高质量。文本提示可能不准确，鼓励用户使用高级视频字幕模型重新标注。数据集的使用遵循ODC-By v1.0许可证，渲染对象的许可证包括CC-BY 4.0、CC-BY-NC 4.0、CC-BY-SA 4.0、CC-BY-NC-SA 4.0和CC0 1.0。
The MV-Video Dataset is a large-scale multi-view video dataset rendered from 53K animated 3D objects. It is used to train the Animate3D model, which can animate any 3D model via multi-view video diffusion technology. For each object, 16 uniformly distributed views are rendered; the elevation angles of the views are randomly sampled within 0–30 degrees, and the initial azimuth angles have random offsets of ±11.25 degrees. The provided videos have a duration of 2 seconds (24 frames per second). For animations lasting 2–4 seconds, only the first 2 seconds are rendered; for animations longer than 4 seconds, both the first 2 seconds and the last 2 seconds are rendered. For objects with more than 6 animations, 6 animations are randomly sampled to avoid overfitting. The dataset consists of multiple compressed files, which contain video files and metadata files after decompression. Portions of the dataset have been filtered, so the actual available quantity is slightly less than that reported in the paper. Approximately 7.7K objects are labeled as having the highest quality. The text prompts may be inaccurate, and users are encouraged to use advanced video captioning models for re-annotation. The use of the dataset is governed by the ODC-By v1.0 license, and the licenses for the rendered objects include CC-BY 4.0, CC-BY-NC 4.0, CC-BY-SA 4.0, CC-BY-NC-SA 4.0 and CC0 1.0.
MV-Video 数据集
概述
MV-Video 是一个大规模的多视角视频数据集，由 53K 个动画 3D 对象渲染而成。该数据集用于训练 Animate3D: Animating Any 3D Model with Multi-view Video Diffusion。
渲染细节
- 每个对象渲染 16 个视图，均匀分布在方位角上。
- 仰角（
elv）在 0-30 度之间随机采样，起始方位角（azi_start）应用 ±11.25 度的随机偏移。 - 每个视频时长为 2 秒（24 fps）。对于 2-4 秒的动画，渲染前 2 秒；对于超过 4 秒的动画，渲染前 2 秒和后 2 秒。
- 对于超过 6 个动画的对象，随机采样 6 个动画以避免过拟合。
数据结构
数据集包含多个 multi_view_video_*.tar.gz 文件，提取后结构如下：
videos/ ├── [UID1]/ │ ├── 00/ │ │ ├── view_0.mp4 │ │ ├── view_1.mp4 │ │ └── ... │ ├── 01/ │ │ ├── view_0.mp4 │ │ ├── view_1.mp4 │ │ └── ... │ └── ... ├── [UID2]/ │ ├── 00/ │ │ ├── view_0.mp4 │ │ ├── view_1.mp4 │ │ └── ... │ └── ... └── ...
- 提供
uid_info_dict.json文件，包含 3D 对象的元信息。
注意事项
- 数据集中约有 500 个动画模型在数据检查过程中被过滤，因此提供的数据略少于论文中报告的数量。
- 标记了约
7.7K个对象为最高质量，列在high_quality_uid.txt文件中。 - 使用 Minigpt4-video 为多视角视频添加文本提示，部分动画的文本提示可能不准确，建议用户使用高级视频字幕模型重新标注。
许可证
数据集使用 ODC-By v1.0 许可证。渲染对象的许可证如下：
- CC-BY 4.0 - 50,000
- CC-BY-NC 4.0 ~ 1,500
- CC-BY-SA 4.0 ~ 400
- CC-BY-NC-SA 4.0 ~ 400
- CC0 1.0 ~ 100
引用
@article{ jiang2024animate3d, title={Animate3D: Animating Any 3D Model with Multi-view Video Diffusion}, author={Yanqin Jiang and Chaohui Yu and Chenjie Cao and Fan Wang and Weiming Hu and Jin Gao}, booktitle={arXiv}, year={2024}, }
