SimScale

<div id="top" align="center">

<p align="center">
<img src="https://ik.imagekit.io/StarBurger/SimScale/title_1080p.gif">
</p>

# **Learning to Drive via Real-World Simulation at Scale**

[](https://github.com/OpenDriveLab/SimScale)
[](https://arxiv.org/abs/2511.23369)
[](https://opendrivelab.com/SimScale/)
[](https://huggingface.co/datasets/OpenDriveLab/SimScale)
[](https://modelscope.cn/datasets/OpenDriveLab/SimScale)
[](https://github.com/OpenDriveLab/SimScale/blob/main/LICENSE)

</div>

<div id="top" align="center">
<p align="center">
<img src="https://raw.githubusercontent.com/OpenDriveLab/SimScale/master/assets/teaser.png" >
</p>
</div>

> [Haochen Tian](https://github.com/hctian713),
> [Tianyu Li](https://github.com/sephyli),
> [Haochen Liu](https://georgeliu233.github.io/),
> [Jiazhi Yang](https://github.com/YTEP-ZHI),
> [Yihang Qiu](https://github.com/gihharwtw),
> [Guang Li](https://scholar.google.com/citations?user=McEfO8UAAAAJ&hl=en),
> [Junli Wang](https://openreview.net/profile?id=%7EJunli_Wang4),
> [Yinfeng Gao](https://scholar.google.com/citations?user=VTn0hqIAAAAJ&hl=en),
> [Zhang Zhang](https://scholar.google.com/citations?user=rnRNwEMAAAAJ&hl=en),
> [Liang Wang](https://scholar.google.com/citations?user=8kzzUboAAAAJ&hl=en),
> [Hangjun Ye](https://scholar.google.com/citations?user=68tXhe8AAAAJ&hl=en),
> [Tieniu Tan](https://scholar.google.com/citations?user=W-FGd_UAAAAJ&hl=en),
> [Long Chen](https://long.ooo/),
> [Hongyang Li](https://lihongyang.info/)
>
>
> - 📧 Primary Contact: Haochen Tian ([e-mail])
> - 📜 Materials: 🌐 [𝕏](https://x.com/OpenDriveLab/status/1999507869633527845) | 📰 [Media](https://mp.weixin.qq.com/s/OGV3Xlb0bHSSSloG11qFJA) | 🗂️ [Slides](https://docs.google.com/presentation/d/17qbsKZU9jdw7MfiPk7hZelaLb3leR2M76gPcMkuf1MI/edit?usp=sharing) | 🎬 [Talk (in Chinese)](https://www.bilibili.com/video/BV1tqrEBNECQ)
> - 🖊️ Joint effort by CASIA, OpenDriveLab at HKU, and Xiaomi EV.

## 🔥 Highlights

- 🏗️ A scalable simulation pipeline that synthesizes diverse and high-fidelity reactive driving scenarios with pseudo-expert demonstrations.
- 🚀 An effective sim-real co-training strategy that improves robustness and generalization synergistically across various end-to-end planners.
- 🔬 A comprehensive recipe that reveals crucial insights into the underlying scaling properties of sim-real learning systems for end-to-end autonomy.

## 📦 Data Preparation

Our released simulation data is based on [nuPlan](https://www.nuscenes.org/nuplan) and [NAVSIM](https://github.com/autonomousvision/navsim). **We recommend first preparing the real-world data by following the instructions in [Download NAVSIM](https://github.com/autonomousvision/navsim/blob/main/docs/install.md#2-download-the-dataset). If you plan to use GTRS, please directly refer [Download NAVSIM](./docs/install.md#2-download-the-dataset).**

### 1. Download Dataset

We provide 🤗 [Script (Hugging Face)](https://github.com/OpenDriveLab/SimScale/blob/main/tools/download_hf.sh) and 👾 [Script (ModelScope)](https://github.com/OpenDriveLab/SimScale/blob/main/tools/download_ms) (users in China) for downloading the simulation data.

Our simulation data format follows that of [OpenScene](https://github.com/OpenDriveLab/OpenScene/blob/main/docs/getting_started.md#download-data), with each clip/log has a fixed temporal horizon of 6 seconds at 2 Hz (2 s history + 4 s future), which are stored separately in `sensor_blobs_hist` and `sensor_blobs_fut`, respectively.
**For policy training, `sensor_blobs_hist` alone is sufficient.**

#### 📊 Overview Table of Simulated Synthetic Data

<table>
<tr style="text-align: center;">
<th rowspan="1">Split / Sim. Round</th>
<th rowspan="1"># Tokens</th>
<th rowspan="1">Logs</th>
<th rowspan="1">Sensors_Hist</th>
<th rowspan="1">Sensors_Fut</th>
<th rowspan="1">Link</th>
</tr>

<th colspan="6">Planner-based Pseudo-Expert</th>

<tr >
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_pdm_v1.0-0.yaml">reaction_pdm_v1.0-0</a></td>
<td>65K</td>
<td>9.9GB</td>
<td>569GB</td>
<td>1.2T</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-0">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-0">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_pdm_v1.0-0">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_pdm_v1.0-1.yaml">reaction_pdm_v1.0-1</a></td>
<td>55K</td>
<td>8.5GB</td>
<td>448GB</td>
<td>964GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-1">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-1">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_pdm_v1.0-1">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_pdm_v1.0-2.yaml">reaction_pdm_v1.0-2</a></td>
<td>46K</td>
<td>6.9GB</td>
<td>402GB</td>
<td>801GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-2">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-2">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_pdm_v1.0-2">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_pdm_v1.0-3.yaml">reaction_pdm_v1.0-3</a></td>
<td>38K</td>
<td>5.6GB</td>
<td>333GB</td>
<td>663GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-3">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-3">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_pdm_v1.0-3">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_pdm_v1.0-4.yaml">reaction_pdm_v1.0-4</a></td>
<td>32K</td>
<td>4.7GB</td>
<td>279GB</td>
<td>554GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-4">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_pdm_v1.0-4">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_pdm_v1.0-4">MS</a></td>
</tr>

<th colspan="6">Recovery-based Pseudo-Expert</th>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_recovery_v1.0-0.yaml">reaction_recovery_v1.0-0</a></td>
<td>45K</td>
<td>6.8GB</td>
<td>395GB</td>
<td>789GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-0">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-0">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_recovery_v1.0-0">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_recovery_v1.0-1.yaml">reaction_recovery_v1.0-1</td>
<td>36K</td>
<td>5.5GB</td>
<td>316GB</td>
<td>631GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-1">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-1">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_recovery_v1.0-1">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_recovery_v1.0-2.yaml">reaction_recovery_v1.0-2</td>
<td>28K</td>
<td>4.3GB</td>
<td>244GB</td>
<td>488GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-2">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-2">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_recovery_v1.0-2">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_recovery_v1.0-3.yaml">reaction_recovery_v1.0-3</td>
<td>22K</td>
<td>3.3GB</td>
<td>189GB</td>
<td>378GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-3">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-3">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_recovery_v1.0-3">MS</a></td>
</tr>

<tr>
<td><a href="https://github.com/OpenDriveLab/SimScale/blob/main/navsim/planning/script/config/common/train_test_split/scene_filter/navtrain_reaction_recovery_v1.0-4.yaml">reaction_recovery_v1.0-4</td>
<td>17K</td>
<td>2.7GB</td>
<td>148GB</td>
<td>296GB</td>
<td><a href="https://huggingface.co/datasets/OpenDriveLab/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-4">HF</a>+
<a href="https://huggingface.co/datasets/OpenDriveLab-org/SimScale/tree/main/SimScale_data/synthetic_reaction_recovery_v1.0-4">HF_Fut</a> /
<a href="https://modelscope.cn/datasets/OpenDriveLab/SimScale/tree/master/SimScale_data/synthetic_reaction_recovery_v1.0-4">MS</a></td>
</tr>

</table>

> [!TIP]
> Before downloading, we recommend checking the table above to select the appropriate split and `sensor_blobs`.

#### 🏭 Simulation Data Pipeline

<div id="top" align="center">
<p align="center">
<img src="https://raw.githubusercontent.com/OpenDriveLab/SimScale/master/assets/pipeline.png" >
</p>
</div>

#### 🧩 Examples of Simulated Synthetic Data

<table align="center">
<tr>
<td align="center">
<p>5c9694f15f9c5537</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/5c9694f15f9c5537.png"/>
</td>
<td align="center">
<p>367cfa28901257ee</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/367cfa28901257ee.png"/>
</td>
<td align="center">
<p>d37c49db3dcd59fa</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/d37c49db3dcd59fa.png"/>
</td>
</tr>
<tr>
<td align="center">
<p>Sim. 1</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/5c9694f15f9c5537-sim1.gif"/><br/>
<p>Sim. 2</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/5c9694f15f9c5537-sim2.gif"/><br/>
<p>Sim. 3</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/5c9694f15f9c5537-sim3.gif"/>
</td>
<td align="center">
<p>Sim. 1</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/367cfa28901257ee-sim1.gif"/><br/>
<p>Sim. 2</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/367cfa28901257ee-sim2.gif"/><br/>
<p>Sim. 3</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/367cfa28901257ee-sim3.gif"/>
</td>
<td align="center">
<p>Sim. 1</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/d37c49db3dcd59fa-sim1.gif"/><br/>
<p>Sim. 2</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/d37c49db3dcd59fa-sim2.gif"/><br/>
<p>Sim. 3</p>
<img src="https://raw.githubusercontent.com/OpenDriveLab/opendrivelab.github.io/master/SimScale/github/d37c49db3dcd59fa-sim3.gif"/>
</td>
</tr>
</table>

### 2. Set Up Configuration
We provide a [Script](https://github.com/OpenDriveLab/SimScale/blob/main/tools/move.sh) for moving the download simulation data to create the following structure.

```angular2html
navsim_workspace/
├── simscale/
├── exp/
└── dataset/
├── maps/
├── navsim_logs/
│ ├── test/
│ ├── trainval/
│ ├── synthetic_reaction_pdm_v1.0-*/
│ │ ├── [log]-00*.pkl
│ │ └── ...
│ └── synthetic_reaction_recovery_v1.0-*/
├── sensor_blobs/
│ ├── test/
│ ├── trainval/
│ ├── synthetic_reaction_pdm_v1.0-*/
│ │ └── [token]-00*/
│ │ ├── CAM_B0/
│ │ └── ...
│ └── synthetic_reaction_recovery_v1.0-*/
└── navhard_two_stage/
```

## ⭐ License and Citation

All content in this repository is under the [Apache-2.0 license](https://www.apache.org/licenses/LICENSE-2.0).
The released data is based on [nuPlan](https://www.nuscenes.org/nuplan) and is under the [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license.

If any parts of our paper and code help your research, please consider citing us and giving a star to our repository.

```bibtex
@article{tian2025simscale,
title={SimScale: Learning to Drive via Real-World Simulation at Scale},
author={Haochen Tian and Tianyu Li and Haochen Liu and Jiazhi Yang and Yihang Qiu and Guang Li and Junli Wang and Yinfeng Gao and Zhang Zhang and Liang Wang and Hangjun Ye and Tieniu Tan and Long Chen and Hongyang Li},
journal={arXiv preprint arXiv:2511.23369},
year={2025}
}
```

搜集汇总

背景与挑战

背景概述

SimScale是一个用于通过大规模真实世界仿真学习驾驶的数据集，基于nuPlan和NAVSIM构建，提供高保真的反应式驾驶场景和伪专家演示。数据集包含历史与未来传感器数据，采用Apache-2.0许可证，由OpenDriveLab发布，旨在支持端到端自动驾驶系统的训练与泛化。