Ximeng0831/CTP-Dataset

---
license: apache-2.0
task_categories:
- zero-shot-image-classification
modalities:
- text
- image
- 3d
tags:
- multi-modal
- point-cloud
- clip
- autonomous-driving
- robotics
- nuscenes
- kitti
size_categories:
- 100K<n<1M
language:
- en
---

# Image-Text-Point Cloud Triplets Dataset
[](https://arxiv.org/abs/2603.07874)
[](https://huggingface.co/Ximeng0831/CTP)
[](https://huggingface.co/datasets/Ximeng0831/CTP-Dataset)
[](https://github.com/TAMU-CVRL/CTP)

This repository contains **Image-Text-Point Cloud triplets** specifically curated for the **Contrastive Tensor Pre-training (CTP)** framework. By aligning 2D visual semantics, 3D LiDAR geometries, and natural language descriptions, this dataset supports research in unified multimodal representation learning.

## Dataset Structure

The dataset is divided by source ([nuScenes](https://www.nuscenes.org/) and [KITTI](https://www.cvlibs.net/datasets/kitti/)).

```text
CTP-Dataset/
├── nuscenes_triplets/
│ ├── nuscenes_image.tar
│ ├── nuscenes_lidar.tar
│ ├── nuscenes_triplet_train.jsonl
│ └── nuscenes_triplet_val.jsonl
└── kitti_triplets/
├── kitti_image.tar
├── kitti_lidar.tar
└── kitti_triplet_train.jsonl
```
### Metadata Format
Each `.jsonl` file contains one JSON object per line, mapping the three modalities:
```json
{
"label": "trafficcone",
"image_path": "nuscenes_image/val/val_0_0_trafficcone.png",
"lidar_path": "nuscenes_lidar/val/val_0_0_trafficcone.npy",
"bbox": [0.966, -5.245, 0.659, 0.291, 0.302, 1.265, 1.551],
"caption": "The traffic cone is orange with a white reflective band near the top, has a conical geometry tapering to a point, and features a black and yellow reflective strip near its base."
}
```

## Quick Start
```python
from huggingface_hub import snapshot_download

# Download datasets

# nuScenes
snapshot_download(
repo_id="Ximeng0831/CTP-Dataset",
repo_type="dataset",
# local_dir="dataset",
allow_patterns="nuscenes_triplets/*"
)

# KITTI
snapshot_download(
repo_id="Ximeng0831/CTP-Dataset",
repo_type="dataset",
# local_dir="dataset",
allow_patterns="kitti_triplets/*"
)
```
Source code: https://github.com/TAMU-CVRL/CTP