KlingTeam/HM-World

---
pretty_name: HM-World
license: apache-2.0
language:
- en
tags:
- video
- computer-vision
- world-model
- multimodal
---

<h1 align="center">Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models</h1>

<div align="center">
<a href="https://kj-chen666.github.io/Hybrid-Memory-in-Video-World-Models/"><img src="https://img.shields.io/badge/Project-Page-orange.svg?logo=googlehome" alt="Project Page"></a>
<a href="https://github.com/H-EmbodVis/HyDRA"><img src="https://img.shields.io/badge/GitHub-Repository-black?logo=github" alt="GitHub"></a>
<a href="https://arxiv.org/pdf/2603.25716"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=Arxiv" alt="arXiv"></a>
</div>

## HM-World

`HM-World` is a dataset for hybrid memory in dynamic video world models. It provides video sequences, foreground masks, camera pose annotations, character pose annotations, event timestamps, and text captions for each sample.

## Usage

Merge the split archives and extract the dataset with:

```bash
cat HM-World_* | tar -xzvf -
```

## Dataset Structure

```text
HM-World/
├── sample1/
│ ├── cond.mp4
│ ├── tgt.mp4
│ ├── cond_mask.mp4
│ ├── tgt_mask.mp4
│ ├── camera.json
│ ├── character.json
│ └── check.json
├── sample2/
│ └── ...
├── ...
└── caption.txt
```

## File Description

- `cond.mp4`: condition video.
- `tgt.mp4`: target video.
- `cond_mask.mp4`: foreground mask video for the condition video.
- `tgt_mask.mp4`: foreground mask video for the target video.
- `camera.json`: camera pose information.
- `character.json`: character pose information.
- `check.json`: timestamps that record when the subjects enters or leaves the frame.
- `caption.txt`: captions for all samples in the dataset.