Waymo-4DSeg

# 🚀 Waymo-4DSeg Dataset
[](https://sam4d-project.github.io/)
[](https://arxiv.org/abs/2506.21547)

## Key features
- **15M** image masks.
- **30M** LiDAR masks.
- **200k** frames.
- **300k** cross-modal masklets.

## Dataset Structure
The data structure are as follows:

```bash
${dataset}
├── meta_infos
│ └── ${sequence_name}.pkl
├── pcds
│ └── ${sequence_name}
│ ├── {timestamp1}.npz
│ ├── {timestamp2}.npz
│ └── ...
├── sam4d_labels (optional)
│ └── ${sequence_name}
│ ├── {timestamp1}.json
│ ├── {timestamp2}.json
│ └── ...
└── undistort_images
└── ${sequence_name}
├── ${timestamp1}
│ ├── ${cam_name}.jpg
│ └── ...
├── ${timestamp2}
│ ├── ${cam_name}.jpg
│ └── ...
└── ...
```

meta_infos: a pickle file containing meta information of the sequence
```python
# meta_infos/${sequence_name}.pkl structure:
from typing import Dict, List, Tuple, Union

MetaInfoType = Dict[str, Union[
str,
List[Dict[str, Union[
Dict[str, Dict[str, Union[
str,
List[List[float]],
None
]]],
Dict[str, str],
List[List[float]]
]]]
]]

example_meta_info: MetaInfoType = {
'seq_name': 'your_sequence_name',
'frames': [
{
'cams_info': {
'your_cam_name': {
'data_path': 'undistort_images/your_sequence_name/your_timestamp/your_cam_name.jpg', # path to image
'camera_intrinsics': [[fx, 0, cx], [0, fy, cy], [0, 0, 1]], # 3x3 matrix
'camera2lidar': [[...], [...], [...], [...]] # 4x4 matrix
},
'your_cam_name2': {...}
},
'path': {
'pcd': 'pcds/your_sequence_name/your_timestamp.npz', # path to point cloud
},
'lidar2world': [[...], [...], [...], [...]] # 4x4 matrix
}
]
}
```

## How to use
After downloading, extract as follows:
```
cat train.tar.gz.part.* | tar -xzf -
cat val.tar.gz.part.* | tar -xzf -
```

# 🚀 Waymo-4DSeg 数据集
[](https://sam4d-project.github.io/)
[](https://arxiv.org/abs/2506.21547)

## 关键特性
- **1500万** 张图像掩码。
- **3000万** 个激光雷达（LiDAR）掩码。
- **20万** 个帧。
- **30万** 个跨模态掩码块（masklets）。

## 数据集结构
数据集结构如下：
bash
${dataset}
├── meta_infos
│ └── ${sequence_name}.pkl
├── pcds
│ └── ${sequence_name}
│ ├── {timestamp1}.npz
│ ├── {timestamp2}.npz
│ └── ...
├── sam4d_labels（可选）
│ └── ${sequence_name}
│ ├── {timestamp1}.json
│ ├── {timestamp2}.json
│ └── ...
└── undistort_images
└── ${sequence_name}
├── ${timestamp1}
│ ├── ${cam_name}.jpg
│ └── ...
├── ${timestamp2}
│ ├── ${cam_name}.jpg
│ └── ...
└── ...

`meta_infos`：存储序列元信息的Pickle文件
python
# meta_infos/${sequence_name}.pkl 文件结构如下：
from typing import Dict, List, Tuple, Union

# 定义序列元信息的类型别名
MetaInfoType = Dict[str, Union[
str,
List[Dict[str, Union[
Dict[str, Dict[str, Union[
str,
List[List[float]],
None
]]],
Dict[str, str],
List[List[float]]
]]]
]]

# 示例元信息
example_meta_info: MetaInfoType = {
'seq_name': 'your_sequence_name',
'frames': [
{
'cams_info': {
'your_cam_name': {
'data_path': 'undistort_images/your_sequence_name/your_timestamp/your_cam_name.jpg', # 图像文件路径
'camera_intrinsics': [[fx, 0, cx], [0, fy, cy], [0, 0, 1]], # 3×3 相机内参矩阵
'camera2lidar': [[...], [...], [...], [...]] # 4×4 相机到激光雷达的变换矩阵
},
'your_cam_name2': {...}
},
'path': {
'pcd': 'pcds/your_sequence_name/your_timestamp.npz', # 点云文件路径
},
'lidar2world': [[...], [...], [...], [...]] # 4×4 激光雷达到世界坐标系的变换矩阵
}
]
}

## 使用方法
下载完成后，请按如下方式解压：

cat train.tar.gz.part.* | tar -xzf -
cat val.tar.gz.part.* | tar -xzf -