WBench

<div align="center"> <img src="assets/longcat-logo-full.png" width="300"> <h1>WBench Dataset</h1> <p><i>A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation</i></p> [](https://huggingface.co/papers/2605.25874) [](https://github.com/meituan-longcat/WBench) [](https://huggingface.co/meituan-longcat/WBench-weights) [](https://modelscope.cn/datasets/meituan-longcat/WBench) [](https://meituan-longcat.github.io/WBench/) </div> --- <div align="center"> <img src="assets/teaser.png" width="90%"> </div> <p align="center" style="color: grey;"> <b>TL;DR</b> — WBench evaluates 20 video world models across 5 dimensions and 22 metrics. </p> ## Overview WBench is a comprehensive multi-turn benchmark for interactive video world model evaluation. It contains **289 multi-turn interaction cases** with 1,058 interaction turns for evaluating models across **22 metrics** and **5 dimensions**: 1. Video Quality 2. Setting Adherence 3. Interaction Adherence 4. Consistency 5. Physics Compliance The benchmark covers diverse scenes, styles, subjects, and both first- and third-person perspectives, with four interaction types: navigation, subject action, event editing, and perspective switching. <div align="center"> <img src="assets/distribution.png" width="90%"> </div> ## Usage ### Download Dataset You can download the dataset using the Hugging Face CLI: ```bash pip install huggingface_hub huggingface-cli download meituan-longcat/WBench --repo-type dataset --local-dir data/ --exclude "splits/*" ``` ### Framework Usage To generate multi-turn videos from a case using the WBench framework (as implemented in the [GitHub repository](https://github.com/meituan-longcat/WBench)): ```python from src.models import get_model # Available: wan, kling, seedance (or register your own) model = get_model("wan") # Generate multi-turn video from a case # case_dict should be loaded from the 'cases/' directory result = model.generate_multi_turn( case=case_dict, output_path="work_dirs/wan/videos/case_1_combined.mp4", data_root="data/", ) ``` ## Structure ``` ├── cases/ # 289 case JSONs (prompts + interactions + annotations) ├── images/ # 289 first-frame images ├── masks/ # 188 subject segmentation masks ``` ## 🏆 Leaderboard **20 Models — Navigation Split (5 Dimensions, sorted by average)** | # | Model | **Average** | Quality | Setting | Interaction | Consistency | Physical | |:---:|:---|:---:|:---:|:---:|:---:|:---:|:---:| | 1 | Kling 3.0 | **79.2 🥇** | 83.0 🥈 | 91.0 🥈 | 70.3 | 82.5 | 69.3 🥉 | | 2 | LingBot-World | **78.8 🥈** | 81.5 | 72.6 | 79.8 | 88.9 🥇 | 71.2 🥈 | | 3 | Wan 2.7 | **78.5 🥉** | 82.6 🥉 | 91.4 🥇 | 66.0 | 80.5 | 71.8 🥇 | | 4 | HY-World 1.5 | **78.4** | 80.2 | 72.2 | 87.5 🥇 | 86.0 | 66.3 | | 5 | HY-Video 1.5 | **78.2** | 79.7 | 85.6 🥉 | 71.8 | 86.7 🥉 | 67.4 | *Detailed leaderboard results and metric breakdowns are available on the [Project Page](https://meituan-longcat.github.io/WBench/#leaderboard).* ## Citation ```bibtex @article{ying2026wbenchcomprehensivemultiturnbenchmark, title={WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation}, author={Ying, Kaining and Hu, Hengrui and Ren, Siyu and Li, Jiamu and Chen, Fengjiao and Wang, Ziwen and Cao, Xuezhi and Cai, Xunliang and Ding, Henghui}, journal={arXiv preprint arXiv:2605.25874}, year={2026} } ```

搜集汇总

背景与挑战

背景概述

WBench 是一个用于评估交互式视频世界模型的综合性多轮基准测试，包含 289 个多轮交互案例，涵盖 5 个维度（如视频质量和一致性）和 22 个指标。该数据集旨在通过多样化的场景和交互类型，全面测试模型的性能。