Waymo-4DSeg

# 🚀 Waymo-4DSeg Dataset [](https://sam4d-project.github.io/) [](https://arxiv.org/abs/2506.21547) ## Key features - **15M** image masks. - **30M** LiDAR masks. - **200k** frames. - **300k** cross-modal masklets. ## Dataset Structure The data structure are as follows: ```bash ${dataset} ├── meta_infos │ └── ${sequence_name}.pkl ├── pcds │ └── ${sequence_name} │ ├── {timestamp1}.npz │ ├── {timestamp2}.npz │ └── ... ├── sam4d_labels (optional) │ └── ${sequence_name} │ ├── {timestamp1}.json │ ├── {timestamp2}.json │ └── ... └── undistort_images └── ${sequence_name} ├── ${timestamp1} │ ├── ${cam_name}.jpg │ └── ... ├── ${timestamp2} │ ├── ${cam_name}.jpg │ └── ... └── ... ``` meta_infos: a pickle file containing meta information of the sequence ```python # meta_infos/${sequence_name}.pkl structure: from typing import Dict, List, Tuple, Union MetaInfoType = Dict[str, Union[ str, List[Dict[str, Union[ Dict[str, Dict[str, Union[ str, List[List[float]], None ]]], Dict[str, str], List[List[float]] ]]] ]] example_meta_info: MetaInfoType = { 'seq_name': 'your_sequence_name', 'frames': [ { 'cams_info': { 'your_cam_name': { 'data_path': 'undistort_images/your_sequence_name/your_timestamp/your_cam_name.jpg', # path to image 'camera_intrinsics': [[fx, 0, cx], [0, fy, cy], [0, 0, 1]], # 3x3 matrix 'camera2lidar': [[...], [...], [...], [...]] # 4x4 matrix }, 'your_cam_name2': {...} }, 'path': { 'pcd': 'pcds/your_sequence_name/your_timestamp.npz', # path to point cloud }, 'lidar2world': [[...], [...], [...], [...]] # 4x4 matrix } ] } ``` ## How to use After downloading, extract as follows: ``` cat train.tar.gz.part.* | tar -xzf - cat val.tar.gz.part.* | tar -xzf - ```# 🚀 Waymo-4DSeg 数据集 [](https://sam4d-project.github.io/) [](https://arxiv.org/abs/2506.21547) ## 关键特性 - **1500万** 张图像掩码。 - **3000万** 个激光雷达（LiDAR）掩码。 - **20万** 个帧。 - **30万** 个跨模态掩码块（masklets）。 ## 数据集结构 数据集结构如下： bash ${dataset} ├── meta_infos │ └── ${sequence_name}.pkl ├── pcds │ └── ${sequence_name} │ ├── {timestamp1}.npz │ ├── {timestamp2}.npz │ └── ... ├── sam4d_labels（可选） │ └── ${sequence_name} │ ├── {timestamp1}.json │ ├── {timestamp2}.json │ └── ... └── undistort_images └── ${sequence_name} ├── ${timestamp1} │ ├── ${cam_name}.jpg │ └── ... ├── ${timestamp2} │ ├── ${cam_name}.jpg │ └── ... └── ... `meta_infos`：存储序列元信息的Pickle文件 python # meta_infos/${sequence_name}.pkl 文件结构如下： from typing import Dict, List, Tuple, Union # 定义序列元信息的类型别名 MetaInfoType = Dict[str, Union[ str, List[Dict[str, Union[ Dict[str, Dict[str, Union[ str, List[List[float]], None ]]], Dict[str, str], List[List[float]] ]]] ]] # 示例元信息 example_meta_info: MetaInfoType = { 'seq_name': 'your_sequence_name', 'frames': [ { 'cams_info': { 'your_cam_name': { 'data_path': 'undistort_images/your_sequence_name/your_timestamp/your_cam_name.jpg', # 图像文件路径 'camera_intrinsics': [[fx, 0, cx], [0, fy, cy], [0, 0, 1]], # 3×3 相机内参矩阵 'camera2lidar': [[...], [...], [...], [...]] # 4×4 相机到激光雷达的变换矩阵 }, 'your_cam_name2': {...} }, 'path': { 'pcd': 'pcds/your_sequence_name/your_timestamp.npz', # 点云文件路径 }, 'lidar2world': [[...], [...], [...], [...]] # 4×4 激光雷达到世界坐标系的变换矩阵 } ] } ## 使用方法 下载完成后，请按如下方式解压： cat train.tar.gz.part.* | tar -xzf - cat val.tar.gz.part.* | tar -xzf -

搜集汇总

背景与挑战

背景概述

Waymo-4DSeg是一个大规模自动驾驶数据集，包含1500万图像掩码、3000万LiDAR掩码、20万帧和30万跨模态掩码片段，数据总量约669.78GB。该数据集采用Apache License 2.0许可，主要用于多模态感知任务。