OpenDriveLab/DriveLM

---
license: cc-by-nc-sa-4.0
viewer: false
---
# **DriveLM:** Driving with **G**raph **V**isual **Q**uestion **A**nswering.

We facilitate `Perception, Prediction, Planning, Behavior, Motion` tasks with human-written reasoning logic as a connection. We propose the task of GVQA to connect the QA pairs in a graph-style structure. To support this novel task, we provide the DriveLM-Data.

DriveLM-Data comprises two distinct components: DriveLM-nuScenes and DriveLM-CARLA. In the case of DriveLM-nuScenes, we construct our dataset based on the prevailing nuScenes dataset. As for DriveLM-CARLA, we collect data from the CARLA simulator. For now, only the training set of DriveLM-nuScenes is publicly available.

## Prepare DriveLM-nuScenes Dataset

Our DriveLM-nuScenes contains a collection of questions and answers. The dataset is named `v1_0_train_nus.json`. We offer a subset of image data that includes all the images used in our DriveLM. You can also download the full nuScenes dataset [HERE](https://www.nuscenes.org/download).

## Usage

1. Download nuScenes subset image data (or full nuScenes dataset) and `v1_0_train_nus.json`.

2. Organize the data structure as follows:

```
DriveLM
├── data/
│ ├── QA_dataset_nus/
│ │ ├── v1_0_train_nus.json
│ ├── nuscenes/
│ │ ├── samples/
```

## License and Citation

This language dataset is licensed under [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). If you use this dataset, please cite our work:

```BibTeX
@article{drivelm_paper2023,
title={DriveLM: Driving with Graph Visual Question Answering},
author={Sima, Chonghao and Renz, Katrin and Chitta, Kashyap and Chen, Li and Zhang, Hanxue and Xie, Chengen and Luo, Ping and Geiger, Andreas and Li, Hongyang},
journal={arXiv preprint arXiv:2312.14150},
year={2023}
}
```

```BibTeX
@misc{drivelm_repo2023,
title={DriveLM: Driving with Graph Visual Question Answering},
author={DriveLM contributors},
howpublished={\url{https://github.com/OpenDriveLab/DriveLM}},
year={2023}
}
```

For more information and updates, please visit our [GitHub repository](https://github.com/OpenDriveLab/DriveLM).

---
许可证：CC-BY-NC-SA-4.0
查看器：禁用
---
# **DriveLM**：融合图视觉问答的自动驾驶
我们以人工撰写的推理逻辑为联结纽带，赋能感知（Perception）、预测（Prediction）、规划（Planning）、行为决策（Behavior）与运动控制（Motion）五类自动驾驶核心任务。为此我们提出了图视觉问答（Graph Visual Question Answering，GVQA）任务，采用图式结构组织问答对。为支撑该全新任务，我们构建了DriveLM-Data数据集。

DriveLM-Data包含两个独立子数据集：DriveLM-nuScenes与DriveLM-CARLA。其中DriveLM-nuScenes基于主流的nuScenes数据集构建而成；而DriveLM-CARLA的数据则来自CARLA仿真平台。目前仅公开了DriveLM-nuScenes的训练集。

## DriveLM-nuScenes数据集准备
本数据集包含多组问答对，存储文件名为`v1_0_train_nus.json`。我们提供了DriveLM任务所需的全部图像数据子集，用户亦可前往[此处](https://www.nuscenes.org/download)下载完整的nuScenes数据集。

## 使用方法
1. 下载nuScenes图像数据子集（或完整nuScenes数据集）与`v1_0_train_nus.json`文件。
2. 按照如下格式组织数据集目录结构：

DriveLM
├── data/
│ ├── QA_dataset_nus/
│ │ ├── v1_0_train_nus.json
│ ├── nuscenes/
│ │ ├── samples/

## 许可证与引用规范
本语言数据集采用[CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)许可证进行授权。若您在研究中使用本数据集，请引用如下文献：
BibTeX
@article{drivelm_paper2023,
title={DriveLM: Driving with Graph Visual Question Answering},
author={Sima, Chonghao and Renz, Katrin and Chitta, Kashyap and Chen, Li and Zhang, Hanxue and Xie, Chengen and Luo, Ping and Geiger, Andreas and Li, Hongyang},
journal={arXiv preprint arXiv:2312.14150},
year={2023}
}

BibTeX
@misc{drivelm_repo2023,
title={DriveLM: Driving with Graph Visual Question Answering},
author={DriveLM contributors},
howpublished={url{https://github.com/OpenDriveLab/DriveLM}},
year={2023}
}

如需获取更多信息与最新动态，请访问我们的[GitHub仓库](https://github.com/OpenDriveLab/DriveLM)。

原始信息汇总

DriveLM: 驾驶中的图视觉问答

数据集概述

DriveLM-Data 包含两个部分：DriveLM-nuScenes 和 DriveLM-CARLA。目前仅公开了 DriveLM-nuScenes 的训练集。

DriveLM-nuScenes 数据集

• 数据文件：v1_0_train_nus.json
• 内容：包含一系列问题和答案。
• 图像数据：提供了一个图像数据子集，包含 DriveLM 中使用的所有图像。

数据结构

数据应组织如下：

DriveLM ├── data/ │ ├── QA_dataset_nus/ │ │ ├── v1_0_train_nus.json │ ├── nuscenes/ │ │ ├── samples/

许可和引用

该数据集遵循 CC-BY-NC-SA 4.0 许可。使用该数据集时，请引用以下文献：

BibTeX @article{drivelm_paper2023, title={DriveLM: Driving with Graph Visual Question Answering}, author={Sima, Chonghao and Renz, Katrin and Chitta, Kashyap and Chen, Li and Zhang, Hanxue and Xie, Chengen and Luo, Ping and Geiger, Andreas and Li, Hongyang}, journal={arXiv preprint arXiv:2312.14150}, year={2023} }

BibTeX @misc{drivelm_repo2023, title={DriveLM: Driving with Graph Visual Question Answering}, author={DriveLM contributors}, howpublished={url{https://github.com/OpenDriveLab/DriveLM}}, year={2023} }

搜集汇总

背景与挑战

背景概述

DriveLM是一个支持图形视觉问答（GVQA）任务的驾驶数据集，包含基于nuScenes和CARLA的两部分数据，涵盖多种驾驶相关任务，并需用户同意共享信息后访问。