TencentARC/VPData

VideoPainter是一个用于任意长度视频修复和编辑的框架，它通过插件式上下文控制，集成了一个高效的上下文编码器。该框架显著降低了模型的学习复杂性，并能够细腻地整合关键的背景上下文。此外，引入了一种目标区域ID重采样技术，实现了任意长度视频的修复，极大地提升了实用性。该框架还建立了一个可扩展的数据管道，利用当前的视觉理解模型，贡献了VPData和VPBench，以促进基于分割的视频修复训练和评估，这是迄今为止最大的视频修复数据集和基准，包含超过390K个多样化的视频片段。

VideoPainter is a framework for any-length video inpainting and editing with plug-and-play context control. It incorporates an efficient context encoder that significantly reduces the models learning complexity while enabling nuanced integration of crucial background context. A novel target region ID resampling technique allows for any-length video inpainting, enhancing practical applicability. The framework also establishes a scalable dataset pipeline leveraging current vision understanding models, contributing VPData and VPBench to facilitate segmentation-based inpainting training and assessment, the largest video inpainting dataset and benchmark to date with over 390K diverse clips.