Eyeline-Labs/Vista4D-Eval-Data

--- license: apache-2.0 language: - en --- # Vista4D: Video Reshooting with 4D Point Clouds (CVPR 2026 Highlight) – Evaluation Dataset [](https://eyeline-labs.github.io/Vista4D) [](https://arxiv.org/abs/2604.21915) [](https://huggingface.co/Eyeline-Labs/Vista4D) [](https://huggingface.co/datasets/Eyeline-Labs/Vista4D-Eval-Data) [Kuan Heng Lin](https://kuanhenglin.github.io)^1,3&lowast;, [Zhizheng Liu](https://bosmallear.github.io)^1,4&lowast;, [Pablo Salamanca](https://pablosalaman.ca)^1,2, [Yash Kant](https://yashkant.github.io)^1,2, [Ryan Burgert](https://ryanndagreat.github.io)^{1,2,5&lowast;}, [Yuancheng Xu](https://yuancheng-xu.github.io)^1,2, [Koichi Namekata](https://kmcode1.github.io)^{1,2,6&lowast;}, [Yiwei Zhao](https://zhaoyw007.github.io)², [Bolei Zhou](https://boleizhou.github.io)⁴, [Micah Goldblum](https://goldblum.github.io)³, [Paul Debevec](https://www.pauldebevec.com)^1,2, [Ning Yu](https://ningyu1991.github.io)^1,2 <br/> ¹Eyeline Labs, ²Netflix, ³Columbia University, ⁴UCLA, ⁵Stony Brook University, ⁶University of Oxford<br> ^&lowast;*Work done during an internship at Eyeline Labs* <div align="center"> <video controls autoplay muted style="width: 100%;" src="https://media.githubusercontent.com/media/Eyeline-Labs/Vista4D/website/media/vista4d.mp4"></video> </div> **Vista4D** is a *video reshooting* framework which synthesizes the dynamic scene represented by an input source video from novel camera trajectories and viewpoints. We bridge the distribution shift between training and inference for point-cloud-grounded video reshooting, as Vista4D is robust to point cloud artifacts from imprecise 4D reconstruction of real-world videos by training on noisy, reconstructed multiview videos. Our 4D point cloud with temporally-persistent static points also explicitly preserves scene content and improved camera control. Vista4D generalizes to real-world applications such as dynamic scene expansion (casual video capture of scene as background reference), 4D scene recomposition (point cloud editing), and long video inference with memory. This is the Hugging Face repository containing our evaluation dataset. We provide 110 video-camera pairs to evaluate Vista4D. We select 13 videos from [DAVIS](https://davischallenge.org/) and 38 videos from [Pexels](https://www.pexels.com/). We use [Pi3](https://yyfz.github.io/pi3/) for 4D reconstruction and [Grounded SAM 2](https://github.com/IDEA-Research/Grounded-SAM-2) to do dynamic pixel segmentation. Then, for each video, we hand-design two to three target cameras for each video using our camera UI. To download the dataset, from the root directory of the project, run ```bash huggingface-cli download Eyeline-Labs/Vista4D-Eval-Data --repo-type dataset --local-dir eval_data ``` to download the Vista4D evaluation dataset into `./eval_data/` and then run ```bash tar -xvf eval_data/eval_data.tar -C eval_data/ ``` to extract the contents. It should have the following structure: ``` eval_data/ metadata.csv recon_and_seg/ # 4D reconstruction and dynamic mask segmentation avocado-slice/ # There should be 51 total videos cameras.npz # Source intrinsics and extrinsics video.mp4 depths/ 00000.exr ... dynamic_mask/ 00000.png ... sky_mask/ # Sky segmentation (to set them to a large depth) 00000.png ... [video_name]/ ... ... cameras/ avocado-slice/ # Two to three target cameras per video close-crane-above.npz left-front-zoom.npz [video_name]/ [camera_name].npz ... ... ``` `metadata.csv` contains the following information: - `name`: Name of video-camera pair, in the format `[video]_[camera]` - `video`: Name of source video, the 4D reconstruction and segmentation can be found in `eval_data/recon_and_seg/[video]/` - `camera`: Name of camera, corresponds to a `video`, can be found in `eval_data/cameras/[video]/[camera].npz` - `seed`: Randomly-generated fixed seed for evaluation - `prompt`: Prompt for the video-camera pair, usually just the prompt of the source video - `dynamic`: Dynamic keywords used to obtain the segmentation map - `do_sky_seg`: Whether the video contains sky (and thus we need to segment it separately) - `source`: Source of the video, `davis` or `pexels` - `video_id`: For videos from `pexels` only, original ID of the video on Pexels, full link is `https://www.pexels.com/video/[video_id]` Instructions on how to use this dataset, model weights, more results, and paper can be found on our [project page](https://eyeline-labs.github.io/Vista4D/) and [GitHub repository](https://github.com/Eyeline-Labs/Vista4D/tree/main).license: Apache 2.0许可证 language: - 英语 # Vista4D：基于4D点云（4D Point Clouds）的视频重拍摄（video reshooting，CVPR 2026 高亮论文）—— 评估数据集 [](https://eyeline-labs.github.io/Vista4D) [](https://arxiv.org/abs/2604.21915) [](https://huggingface.co/Eyeline-Labs/Vista4D) [](https://huggingface.co/datasets/Eyeline-Labs/Vista4D-Eval-Data) [Kuan Heng Lin](https://kuanhenglin.github.io)^1,3&lowast;, [Zhizheng Liu](https://bosmallear.github.io)^1,4&lowast;, [Pablo Salamanca](https://pablosalaman.ca)^1,2, [Yash Kant](https://yashkant.github.io)^1,2, [Ryan Burgert](https://ryanndagreat.github.io)^{1,2,5&lowast;}, [Yuancheng Xu](https://yuancheng-xu.github.io)^1,2, [Koichi Namekata](https://kmcode1.github.io)^{1,2,6&lowast;}, [Yiwei Zhao](https://zhaoyw007.github.io)², [Bolei Zhou](https://boleizhou.github.io)⁴, [Micah Goldblum](https://goldblum.github.io)³, [Paul Debevec](https://www.pauldebevec.com)^1,2, [Ning Yu](https://ningyu1991.github.io)^1,2 <br/> ¹Eyeline Labs, ²Netflix(奈飞), ³Columbia University(哥伦比亚大学), ⁴UCLA(加州大学洛杉矶分校), ⁵Stony Brook University(石溪大学), ⁶University of Oxford(牛津大学)<br> ^&lowast;*工作于Eyeline Labs实习期间完成* <div align="center"> <video controls autoplay muted style="width: 100%;" src="https://media.githubusercontent.com/media/Eyeline-Labs/Vista4D/website/media/vista4d.mp4"></video> </div> **Vista4D**是一款**视频重拍摄（video reshooting）**框架，可基于输入源视频合成动态场景，并支持全新的相机轨迹与视角生成。针对基于点云的视频重拍摄任务中训练与推理的分布偏移问题，本框架通过在带噪声的重建多视图视频上进行训练，对真实世界视频的不精确4D重建产生的点云伪影具备强鲁棒性。此外，我们的4D点云包含时间持久的静态点，可显式保留场景内容并优化相机操控性。Vista4D可泛化至多种真实世界应用，例如动态场景扩展（以随手拍摄的场景视频作为背景参考）、4D场景重组（点云编辑）以及带记忆的长视频推理。 本Hugging Face仓库包含我们的评估数据集。我们提供110组视频-相机对用于Vista4D的评估。数据集源自13段[DAVIS](https://davischallenge.org/)视频与38段[Pexels](https://www.pexels.com/)视频。我们使用[Pi3](https://yyfz.github.io/pi3/)进行4D重建，并使用[Grounded SAM 2](https://github.com/IDEA-Research/Grounded-SAM-2)完成动态像素分割。随后，我们针对每段视频通过自研相机交互界面手动设计2至3个目标相机。 如需下载数据集，请在项目根目录执行以下命令： bash huggingface-cli download Eyeline-Labs/Vista4D-Eval-Data --repo-type dataset --local-dir eval_data 将Vista4D评估数据集下载至`./eval_data/`目录，随后执行： bash tar -xvf eval_data/eval_data.tar -C eval_data/ 解压文件内容。数据集的目录结构如下： eval_data/ metadata.csv recon_and_seg/ # 4D重建与动态掩码分割结果 avocado-slice/ # 总计包含51段视频 cameras.npz # 相机内参与外参 video.mp4 depths/ 00000.exr ... dynamic_mask/ 00000.png ... sky_mask/ # 天空分割掩码（用于设置超大深度值） 00000.png ... [video_name]/ ... ... cameras/ avocado-slice/ # 每段视频对应2至3个目标相机 close-crane-above.npz left-front-zoom.npz [video_name]/ [camera_name].npz ... ... `metadata.csv`包含以下信息： - `name`：视频-相机对的名称，格式为`[video]_[camera]` - `video`：源视频名称，其4D重建与分割结果可在`eval_data/recon_and_seg/[video]/`路径下找到 - `camera`：相机名称，对应某一段视频，其参数可在`eval_data/cameras/[video]/[camera].npz`路径下找到 - `seed`：用于评估的随机固定种子 - `prompt`：该视频-相机对的提示词，通常与源视频的提示词一致 - `dynamic`：用于获取分割掩码的动态关键词 - `do_sky_seg`：该视频是否包含天空（即是否需要单独进行天空分割） - `source`：视频来源，取值为`davis`或`pexels` - `video_id`：仅针对Pexels来源的视频，为其在Pexels平台的原始视频ID，完整链接为`https://www.pexels.com/video/[video_id]` 有关本数据集的使用说明、模型权重、更多实验结果以及论文详情，请参阅我们的[项目页面](https://eyeline-labs.github.io/Vista4D/)与[GitHub仓库](https://github.com/Eyeline-Labs/Vista4D/tree/main)。