KlingTeam/HM-World

--- pretty_name: HM-World license: apache-2.0 language: - en tags: - video - computer-vision - world-model - multimodal --- <h1 align="center">Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models</h1> <div align="center"> <a href="https://kj-chen666.github.io/Hybrid-Memory-in-Video-World-Models/"><img src="https://img.shields.io/badge/Project-Page-orange.svg?logo=googlehome" alt="Project Page"></a> <a href="https://github.com/H-EmbodVis/HyDRA"><img src="https://img.shields.io/badge/GitHub-Repository-black?logo=github" alt="GitHub"></a> <a href="https://arxiv.org/pdf/2603.25716"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=Arxiv" alt="arXiv"></a> </div> ## HM-World `HM-World` is a dataset for hybrid memory in dynamic video world models. It provides video sequences, foreground masks, camera pose annotations, character pose annotations, event timestamps, and text captions for each sample. ## Usage Merge the split archives and extract the dataset with: ```bash cat HM-World_* | tar -xzvf - ``` ## Dataset Structure ```text HM-World/ ├── sample1/ │ ├── cond.mp4 │ ├── tgt.mp4 │ ├── cond_mask.mp4 │ ├── tgt_mask.mp4 │ ├── camera.json │ ├── character.json │ └── check.json ├── sample2/ │ └── ... ├── ... └── caption.txt ``` ## File Description - `cond.mp4`: condition video. - `tgt.mp4`: target video. - `cond_mask.mp4`: foreground mask video for the condition video. - `tgt_mask.mp4`: foreground mask video for the target video. - `camera.json`: camera pose information. - `character.json`: character pose information. - `check.json`: timestamps that record when the subjects enters or leaves the frame. - `caption.txt`: captions for all samples in the dataset.