TrajScene-60K

# [ECCV 2026] MoGe4D: Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation <b>[Yanran Zhang](https://github.com/Zhangyr2022/)^\*,1, [Ziyi Wang](https://wangzy22.github.io/)^\*,1, [Wenzhao Zheng](https://wzzheng.net/#)^†,1, [Zheng Zhu](http://www.zhengzhu.net/)², [Jie Zhou](https://scholar.google.com/citations?user=6a79aPwAAAAJ&hl=en)¹, [Jiwen Lu](https://ivg.au.tsinghua.edu.cn/Jiwen_Lu/)¹</b> ¹Department of Automation, Tsinghua University &nbsp;&nbsp;&nbsp; ²GigaAI <i>^*Equal Contribution &nbsp;&nbsp; ^†Corresponding Author</i> <p align="center"> <a href="https://github.com/Zhangyr2022/MoGe4D"><img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="GitHub"></a> <a href="https://arxiv.org/abs/2512.05044"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a> <a href="https://ivg-yanranzhang.github.io/MoGe4D/"><img src="https://img.shields.io/badge/Project-Website-blue?logo=googlechrome&logoColor=white" alt="Project"></a> <br> <a href="https://www.modelscope.cn/models/YanranZhang/MoGe4D"><img src="https://img.shields.io/badge/🤖%20ModelScope-Model-4e29ff" alt="ModelScope Model"></a> <a href="https://huggingface.co/Yanran21/MoGe4D"><img src="https://img.shields.io/badge/🤗%20HuggingFace-Model-ffd21e" alt="HuggingFace Model"></a> <a href="https://www.modelscope.cn/datasets/YanranZhang/TrajScene-60K"><img src="https://img.shields.io/badge/🤖%20ModelScope-Dataset-4e29ff" alt="ModelScope Dataset"></a> </p> ## 📄 Paper Summary A robust 4D generative framework needs a multi-faceted dataset with three modalities: **dense 4D point trajectories**, **viewpoint-specific visual observations**, and **high-level semantic descriptions**. To address the acute scarcity of large-scale, high-quality 4D scene data with complex dynamics, we build **TrajScene-60K**. It is curated from the [WebVid-10M](https://github.com/m-bain/webvid) corpus: ~200,000 candidates are extracted and filtered with VLM/LLM-based caption filtering (CogVLM2 & DeepSeek-V3), followed by scale-consistency and geometry validation. The result is a real-world, indoor/outdoor dataset with dense occlusion-aware tracking, per-frame depth, and language descriptions — significantly surpassing prior 4D datasets in scale and annotation richness. ## 📊 Statistics | Statistic | Value | |---|---| | Samples | 60,000 videos | | Total frames | 3M+ | | 3D point annotations | ~12 billion (1.2 × 10¹⁰) | | Resolution | 596 × 336, 49 frames | | Source | WebVid-10M (VLM-filtered) | | Annotations | dense 4D point trajectories · per-frame depth · occlusion masks · captions | | Scene type | real-world, indoor & outdoor | ## 📁 Directory Structure The dataset is shipped as range archives `<range>.tar.gz` (50 samples per range), covering all 60,000 samples (scenes `000001`–`060000`). ``` <range>.tar.gz # e.g. 000001_000050.tar.gz (scenes 000001–000050) └── <range>/ ├── <id>.mp4 # source video (ground-truth observation) ├── <id>.txt # caption (scene content + dynamic behavior) ├── <id>_dt3d_pred.pkl # dense 4D point trajectories ├── <id>_dt3d_render.mp4 # rendered multi-view video ├── <id>_mask_render.mp4 # occlusion mask video └── <id>_mask_render_binary.npy # binary occlusion mask ``` > Range archives are numbered continuously: `000001_000050`, `000051_000100`, …, `059951_060000`. A small number of ranges with no valid renderings are omitted. ## ⬇️ Download ```bash pip install modelscope modelscope download --dataset YanranZhang/TrajScene-60K --local_dir ./data/TrajScene-60K ``` Each archive is self-contained — you can download a subset of range archives for quick experiments. ## 🛠️ Usage TrajScene-60K provides the supervision to train MoGe4D's components: - **4D point trajectories (`*_dt3d_pred.pkl`)** + depth → train **4D-STraG** (trajectory generation with depth-guided motion normalization). - **Rendered/mask videos** + source videos + captions → train **4D-ViSM** (novel-view synthesis). See the [GitHub repository](https://github.com/Zhangyr2022/MoGe4D) (`scripts/4D_STraG_training`, `scripts/4D_ViSM_training`) for the data-loading and training pipeline. ## ⚖️ License & Disclaimer Released under the Apache 2.0 license for research purposes. TrajScene-60K is derived from WebVid-10M using LLM/VLM-based filtering and therefore inherits the demographic and content biases of the source videos and filtering models. Users are responsible for complying with the original WebVid-10M license and any applicable regulations. ## 📖 Citation ```bibtex @inproceedings{zhang2026moge4d, title={Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation}, author={Zhang, Yanran and Wang, Ziyi and Zheng, Wenzhao and Zhu, Zheng and Zhou, Jie and Lu, Jiwen}, booktitle={European Conference on Computer Vision (ECCV)}, year={2026} } ``` ## 📧 Contact - Yanran Zhang — [e-mail]