RoVid-X

<p align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/6661917459720067b2a15bd6/zQIVX2Yjqb0Goi8irZZ7D.png" width="200px" /> </p> <h2 align="center" style="font-size: 30px; font-weight: bold;"> <a href="https://dagroup-pku.github.io/ReVidgen.github.io/">Rethinking Video Generation Model for the Embodied World</a> </h2> <div align="center"> <a href="https://github.com/DAGroup-PKU/ReVidgen"> <img src="https://img.shields.io/badge/GitHub-181717?style=for-the-badge&logo=github"/> </a> <a href="https://arxiv.org/abs/2601.15282"> <img src="https://img.shields.io/badge/Paper-4B0082?style=for-the-badge&logo=arxiv"/> </a> <a href="https://dagroup-pku.github.io/ReVidgen.github.io/"> <img src="https://img.shields.io/badge/Project-2E8B57?style=for-the-badge&logo=googlechrome"/> </a> <a href="https://huggingface.co/datasets/DAGroup-PKU/RoVid-X"> <img src="https://img.shields.io/badge/Dataset-005FED?style=for-the-badge&logo=huggingface"/> </a> <a href="https://huggingface.co/datasets/DAGroup-PKU/RBench"> <img src="https://img.shields.io/badge/Benchmark-FF8C00?style=for-the-badge&logo=semanticweb"/> </a> <a href="https://huggingface.co/spaces/DAGroup-PKU/RBench-Leaderboard"> <img src="https://img.shields.io/badge/Leaderboard-CDA434?style=for-the-badge&logo=googlesheets"/> </a> <a href="https://youtu.be/Ea91ErBMBLM"> <img src="https://img.shields.io/badge/Video-CC0000?style=for-the-badge&logo=youtube"/> </a> </div> <h5 align="center"> If you like our project, please give us a star ⭐ on GitHub for the latest update. </h5> ## Key features - **4M robotic video clips(10K+ hours)** for large-scale video generation training. - **1300+ fine-grained robotic skills**, covering diverse actions and task primitives. - **Multi-modal physical annotations**, including **RGB, depth, and optical flow**. - **Multi-robot and multi-task diversity**, spanning various robot types, scenarios, and action skills. - **Rich object interactions**, enabling complex and realistic robot behavior modeling. ## Dataset Structure RoVid-X provides structured annotations for each video clip in JSON format, where each entry is indexed by the video filename. ```bash 📄 JSON Example { "5bebd280785add39d0e40cbea9e0c545.mp4": { "verb": "grab@@place", "task_caption": "grab the tape and place it into the basket", "short_caption": "The robotic arm moves toward the tape, grasps it, and places it into the basket.", "detailed_caption": "The video opens with an overhead view of a table covered with a red-and-white checkered tablecloth in a room equipped with technical devices. On the table..." } } ``` ## Download You can download **RoVid-X** directly from Hugging Face using the official CLI. ```bash # (Optional) If you are in mainland China, use the Hugging Face mirror export HF_ENDPOINT=https://hf-mirror.com # Download the dataset huggingface-cli download DAGroup-PKU/RoVid-X --local-dir datasets/RoVid-X ``` ## 📚 Citation If you find this dataset useful, please cite our paper: ```bibtex @article{deng2026rethinking, title={Rethinking Video Generation Model for the Embodied World}, author={Deng, Yufan and Pan, Zilin and Zhang, Hongyu and Li, Xiaojie and Hu, Ruoqing and Ding, Yufei and Zou, Yiming and Zeng, Yan and Zhou, Daquan}, journal={arXiv preprint arXiv:2601.15282}, year={2026} }