Vera-Layered-Video-Dataset

# Dataset for Vera: A Layered Diffusion Model for Content-Preserving Video Editing <p align="left"> <a href="https://devzhk.github.io/">Hongkai Zheng</a>¹²* &nbsp;&middot;&nbsp; <a href="https://ttchengab.github.io/">Ta-Ying Cheng</a>² &nbsp;&middot;&nbsp; <a href="https://scholar.google.com/citations?user=xkX9W9QAAAAJ&hl=en">Benjamin Klein</a>² &nbsp;&middot;&nbsp; <a href="https://yisongyue.com/">Yisong Yue</a>¹ &nbsp;&middot;&nbsp; <a href="https://zhuoning.cc/">Zhuoning Yuan</a>²† </p> <p align="left"> ¹California Institute of Technology &nbsp;&nbsp; ²Netflix, Inc.<br> _{*Work done during an internship at Netflix &nbsp; †Project Lead} </p> <p align="left"> <a href="https://vera-layered-diffusion.github.io/"><img src="https://img.shields.io/badge/🌐-Project%20Page-blue" alt="Project Page"></a> <a href="https://arxiv.org/abs/2606.23610"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="Paper"></a> <a href="https://huggingface.co/datasets/netflix/Vera-Layered-Video-Dataset"><img src="https://img.shields.io/badge/🤗%20Hugging%20Face-Dataset-ff9d00" alt="Dataset"></a> <a href="#"><img src="https://img.shields.io/badge/License-Apache%202.0-green.svg" alt="License"></a> </p> > **TL;DR**: A layered diffusion framework for video editing. Vera jointly generates an edit layer, an alpha matte, and a composite video, separating what to generate from what to preserve. > > **Disclaimer:** This is a research prototype, not an official product. <p align="left"> <video src="https://vera-layered-diffusion.github.io/assets/videos/vera-promo-video-v5-0.7x.mp4" width="50%" controls autoplay loop muted playsinline></video> </p> ## 📋 Dataset Description - **Curated by:** [Hongkai Zheng](https://devzhk.github.io/), [Ta-Ying Cheng](https://ttchengab.github.io/), [Benjamin Klein](https://scholar.google.com/citations?user=xkX9W9QAAAAJ&hl=en), [Yisong Yue](https://yisongyue.com/), [Zhuoning Yuan](https://zhuoning.cc/) - **License:** [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) - **Paper:** [Vera: A Layered Diffusion Model for Content-Preserving Video Editing](https://arxiv.org/abs/2606.23610) --- ## 📦 Dataset Structure ### 🎞️ Splits — 49 Frames (3 sec) **Note:** The current Vera models are trained on 49-frame sequences. | Split | Edit Type | # Samples | | ---------------------------------------------- | ------------------ | ---------- | | train / 49-frames / realistic-set1-bg-change | background_replace | 914 | | train / 49-frames / realistic-set1-obj-add | obj_add | 470 | | train / 49-frames / realistic-set2-obj-add | obj_add | 770 | | train / 49-frames / synthetic-bg-change | background_replace | 4,994 | | train / 49-frames / synthetic-obj-add | obj_add | 4,848 | | **49-Frame Train Total** | | **11,996** | ### 🎞️ Splits — 81 Frames (5 sec) | Split | Edit Type | # Samples | | ---------------------------------------------- | ------------------ | ---------- | | train / 81-frames / realistic-set1-bg-change | background_replace | 457 | | train / 81-frames / realistic-set1-obj-add | obj_add | 235 | | train / 81-frames / realistic-set2-obj-add | obj_add | 385 | | train / 81-frames / synthetic-bg-change | background_replace | 2,497 | | train / 81-frames / synthetic-obj-add | obj_add | 2,431 | | **81-Frame Train Total** | | **6,005** | ### 🧪 Test Splits | Split | Edit Type | # Samples | | ----------------- | ------------------ | --------- | | test / bg-change | background_replace | 69 | | test / obj-add | obj_add | 72 | | **Test Total** | | **141** | --- ## 🗂️ Data Sources ### 🏋️ Training Set | Source | License | | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | [Pexels](https://www.pexels.com/) | [Pexels License](https://www.pexels.com/license/) | | [Mixkit](https://mixkit.co/) | [Mixkit License](https://mixkit.co/license/) | | [VideoMatte240K](https://grail.cs.washington.edu/projects/background-matting-v2/) | [MIT License](https://github.com/PeterL1n/BackgroundMattingV2/issues/208) | ### 🧪 Test Set The test set is sourced from the training sources above, plus: | Source | License | | --------------------------------------------------- | ----------------------------------------------------------------- | | [DAVIS](https://davischallenge.org/) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) | | [VACEBench](https://ali-vilab.github.io/VACE-page/) | [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) | --- ## 📝 Citation ```bibtex @article{zheng2026vera, title = {Vera: A Layered Diffusion Model for Content-Preserving Video Editing}, author = {Zheng, Hongkai and Cheng, Ta-Ying and Klein, Benjamin and Yue, Yisong and Yuan, Zhuoning}, journal = {arXiv preprint arXiv:2606.23610}, year = {2026} } ```