theairlabcmu/TartanRGBT

---
license: bsd-3-clause-clear
tags:
- thermal
- multi-modal
- robotics
pretty_name: TartanRGBT
viewer: false
---
# TartanRGBT Dataset

## Overview

**TartanRGBT** is a large-scale, synchronized **RGB–Thermal–Depth dataset** collected across diverse **indoor, outdoor, urban, off-road, and park environments**.
The dataset is designed to support research in **robot perception**, **visual localization**, **cross-modal representation learning**, **thermal vision**, and **multi-sensor fusion**.

All data is organized by **day → trajectory → modality**, and distributed as **ZIP archives per modality** to enable efficient storage, selective download, and scalable use.

This dataset is part of the work -

If you use this dataset, please cite our work : AnyThermal: Towards Learning Universal Representations for Thermal Perception. To know more about this project, please visit our [project website](https://anythermal.github.io/)

@misc{maheshwari2026anythermallearninguniversalrepresentations,
title={AnyThermal: Towards Learning Universal Representations for Thermal Perception},
author={Parv Maheshwari and Jay Karhade and Yogesh Chawla and Isaiah Adu and Florian Heisen and Andrew Porco and Andrew Jong and Yifei Liu and Santosh Pitla and Sebastian Scherer and Wenshan Wang},
year={2026},
eprint={2602.06203},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2602.06203},
}

## Sensors & Modalities

Each trajectory contains the following modalities (provided as separate ZIP files):

- `thermal_left_rect_8.zip`
- `thermal_right_rect_8.zip`
- `thermal_right_rect_16.zip`
- `thermal_right_left_16.zip`
- `zed_left_rect.zip`
- `zed_right_rect.zip`
- `RGB_aligned_with_thermal.zip`
- `stereo_depth.zip`

All sequences are temporally aligned across modalities and sampled at **1 Hz**.

## Dataset Structure

After download and extraction, the dataset follows the structure below:

```text
TartanRGBT_dataset/
├── day1/
│ └── /
│ ├── thermal_left_rect_8/
│ │ ├── 000000.png
│ │ ├── 000001.png
│ │ └── ...
│ ├── thermal_right_rect_8/
│ ├── thermal_right_rect_16/
│ ├── thermal_right_left_16/
│ ├── zed_left_rect/
│ ├── zed_right_rect/
│ ├── RGB_aligned_with_thermal/
│ └── stereo_depth/
│ └── [*.zip] # Optional - if --delete_zips=no
├── day2/
├── day3/
├── day4/
└── day5/
```

On Hugging Face, **only ZIP files are stored**, preserving the same hierarchy: dayX/trajectory_name/*.zip

## Requirements
```text
pip install huggingface_hub
```
## Extraction Script

The extraction script can be found at the root of this repository as data_extraction.py. The script performs the following steps:

1. Downloads all ZIP files from the theairlabcmu/TartanRGBT repository to a local directory
2. Extracts each ZIP file in place
3. (Optional) Deletes ZIP files after successful extraction

Usage
```text
python data_extraction.py <base_dir> [--delete_zips yes|no]
```
### Arguments
- `base_dir`: Directory where `TartanRGBT_dataset/` folder will be created
- `--delete_zips`: Whether to delete ZIP files after extraction (default: `no`)