Mona4399/CARLA_OOD

---
task_categories:
- image-segmentation
size_categories:
- n<1K
---
# Dataset Card for Carla_OOD Dataset

## Dataset Summary

The Carla_OOD Dataset, created using the CARLA v0.9.13 simulator, mimics the sensor configuration of the KITTI dataset with a Velodyne HDL64 LiDAR and cameras aligned with KITTI's Camera0. It is designed to advance research in anomaly detection and segmentation in autonomous systems, addressing challenges in handling unexpected multi-modal inputs.

## Supported Tasks

The Carla_OOD Dataset can be used to compare research approaches for detecting or segmenting anomalies in the input. It therefore bridges the gap towards deploying learning systems in autonomous vehicles, which by definition must handle unexpected multi-modal (LiDAR and camera) inputs and anomalies.

## Dataset Instance

<div align="center">
<figure style="display: inline-block; margin: 20px; width: 30%;">
<img src="Examples/0003_0.png" alt="RGB Image" style="width: 100%;">
<figcaption>RGB Image</figcaption>
</figure>
<figure style="display: inline-block; margin: 20px; width: 30%;">
<img src="Examples/0003_10.png" alt="Semantic Image" style="width: 100%;">
<figcaption>Semantic Image</figcaption>
</figure>
<figure style="display: inline-block; margin: 20px; width: 30%;">
<img src="Examples/0003.png" alt="Lidar Back Projected onto 2D Image" style="width: 100%;">
<figcaption>Lidar Back Projected onto 2D Image</figcaption>
</figure>
</div>

## Dataset Details

### Sensor Configuration
The dataset utilizes sensors configured as follows to simulate real-world autonomous driving scenarios:

- **Semantic LIDAR Sensor**:
- **Location**: `carla.Location(x=0, y=0, z=1.80)`
- **Rotation**: `carla.Rotation(pitch=0, yaw=0, roll=0)`
- **channels**: `64`
- **range**: `80.0`
- **points_per_second**: `64/0.00004608`
- **rotation_frequency**: `10`
- **upper_fov**: `2`
- **lower_fov**: `-24.8`

- **RGB Camera**:
- **Location**: `carla.Location(x=0.50, y=0, z=1.70)`
- **Rotation**: `carla.Rotation(pitch=0, yaw=0, roll=0)`
- **image_size_x**: `1392`
- **image_size_y**: `1024`
- **fov**: `72`
- **sensor_tick**: `0.10`

- **Semantic Segmentation Camera**:
- **Location**: `carla.Location(x=0.50, y=0, z=1.70)`
- **Rotation**: `carla.Rotation(pitch=0, yaw=0, roll=0)`
- **image_size_x**: `1392`
- **image_size_y**: `1024`
- **fov**: `72`
- **sensor_tick**: `0.10`

### Dataset Structure

The dataset is organized into multiple sequences (e.g., different towns or scenarios). Each sequence contains data under different weather conditions and includes multiple modalities:

- **seq_0/**
- `images_rgb/`
- `0000_0.png`
- ...
- `images_ss/`
- `0000_10.png`
- ...
- `lidar_ss/`
- `0000.ply`
- ...
- `test_proj/`
- `0000.png`
- ...

- **seq_1/**
- **seq_2/**
- **...**
- **seq_21/**

- `lidar_to_cam0.txt`: calibration file for LiDAR-to-camera transformation.

Each sequence corresponds to a specific weather condition and contains RGB images, semantic segmentation maps, LiDAR point clouds, LiDAR projection results, and reference images without obstacles.

### Dataset Class IDs

Class IDs for the semantic LiDAR/Camera labels can be found detailed in the [CARLA Semantic tags](https://carla.readthedocs.io/en/0.9.13/ref_sensors/#:~:text=The%20following-,tags,-are%20currently%20available). The dataset includes 22 classes, each associated with a unique color code as follows:

| Class ID | Tag | Converted color |
|:----------:|:--------------:|:-----------------:|
| 0 | Unlabeled | [0, 0, 0] |
| 1 | Building | [70, 70, 70] |
| 2 | Fence | [100, 40, 40] |
| 3 | Other | [55, 90, 80] |
| 4 | Pedestrian | [220, 20, 60] |
| 5 | Pole | [153, 153, 153] |
| 6 | Road Line | [157, 234, 50] |
| 7 | Road | [128, 64, 128] |
| 8 | Sidewalk | [244, 35, 232] |
| 9 | Vegetation | [107, 142, 35] |
| 10 | Vehicles | [0, 0, 142] |
| 11 | Wall | [102, 102, 156] |
| 12 | Traffic Sign | [220, 220, 0] |
| 13 | Sky | [70, 130, 180] |
| 14 | Ground | [81, 0, 81] |
| 15 | Bridge | [150, 100, 100] |
| 16 | Rail Track | [230, 150, 140] |
| 17 | Guard Rail | [180, 165, 180] |
| 18 | Traffic Light | [250, 170, 30] |
| 19 | Static | [110, 190, 160] |
| 20 | Dynamic | [170, 120, 50] |
| 21 | Water | [45, 60, 150] |
| 22 | Terrain | [145, 170, 100] |

In this dataset, objects belonging to the `Dynamic` and `Static` classes are considered out-of-distribution (OOD) classes.

## Citation

If you find our work useful in your research please consider citing our [paper](https://arxiv.org/abs/2505.16985):

```
@article{liu2025fm,
title={Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation},
author={Liu, Moru and Dong, Hao and Kelly, Jessica and Fink, Olga and Trapp, Mario},
journal={arXiv preprint arXiv:2505.16985},
year={2025}
}
```

**References:**

https://carla.readthedocs.io/en/0.9.13/

https://www.cvlibs.net/datasets/kitti/

https://github.com/jedeschaud/kitti_carla_simulator

## Dataset Card Contact

[[e-mail]]