LingBot-Depth-Dataset

# LingBot-Depth Dataset

Self-curated RGB-D dataset for training [LingBot-Depth](https://github.com/Robbyant/lingbot-depth), a depth estimation model. Each sample contains an RGB image, raw sensor depth, and ground truth depth.

<p align="center">
<img src="teaser.png" width="100%">
</p>

## Sub-datasets

| Name | Description | Samples |
|------|-------------|--------:|
| **RobbyReal** | Real-world indoor scenes captured with multiple RGB-D cameras | 1,400,000 |
| **RobbyVla** | Real-world data collected during VLA (Vision-Language-Action) robot manipulation tasks | 580,960 |
| **RobbySim** | Simulated data rendered from two camera viewpoints | 999,264 |
| **RobbySimVal** | Validation split of simulated data | 38,976 |
| **Total** | | **3,019,200** |

## Directory Structure

### RobbyReal

```
RobbyReal/
└── <scene_id>/ # e.g. 00001_01_room
├── orbbec_335_<seq>/
│ ├── color/ # RGB images
│ ├── gtdepth/ # Ground truth depth
│ ├── rawdepth/ # Raw sensor depth
│ └── intrinsic.txt
├── orbbec_335L_<seq>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
├── realsense_D415_<seq>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
├── realsense_D435_<seq>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── realsense_D455_<seq>/
├── color/
├── gtdepth/
├── rawdepth/
└── intrinsic.txt
```

### RobbyVla

Data collected during VLA robot manipulation tasks on two robot platforms.

```
RobbyVla/
├── franka/ # Franka robot arm
│ └── <seq>/
│ ├── left_realsense405/
│ │ ├── color/
│ │ ├── gtdepth/
│ │ ├── rawdepth/
│ │ └── intrinsic.txt
│ └── right_realsense405/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── ur7e/ # UR7e robot arm
└── <seq>/
├── left_realsense405/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── right_realsense405/
├── color/
├── gtdepth/
├── rawdepth/
└── intrinsic.txt
```

### RobbySim

Simulated data rendered from two camera viewpoints: `object_view` and `rrt_view`.

```
RobbySim/
├── intrinsics.txt
├── object_view/
│ └── <scene_id>/<cam_id>/ # e.g. xxxx/00, xxxx/01
│ ├── 0001_cam0_left.jpg # RGB image
│ ├── 0001_cam0_depth.png # Ground truth depth
│ └── 0001_cam0_rmd2c.png # Raw depth
└── rrt_view/
└── <scene_id>/<cam_id>/
├── 0001_cam0_left.jpg
├── 0001_cam0_depth.png
└── 0001_cam0_rmd2c.png
```

### RobbySimVal

Validation split of simulated data.

```
RobbySimVal/
└── val_view/
└── <scene_id>/<cam_id>/ # e.g. xxxx/00, xxxx/01
├── intrinsics.txt
├── 0001_cam0_rgb.left.jpg # RGB image
├── 0001_cam0_depth_left.png # Ground truth depth
└── 0001_cam0_rawdepth.left.png # Raw depth
```

## File Description

| File/Folder | Description |
|-------------|-------------|
| `color/` / `*_rgb*.jpg` | RGB images |
| `gtdepth/` / `*_depth*.png` | Ground truth depth maps (16-bit PNG, unit: mm) |
| `rawdepth/` / `*_rawdepth*.png` / `*_rmd2c*.png` | Raw depth from sensor before post-processing (16-bit PNG, unit: mm) |
| `intrinsic.txt` / `intrinsics.txt` | Camera intrinsic parameters |

# LingBot-Depth 数据集

本数据集为自研RGB-D数据集，用于训练深度估计模型LingBot-Depth（项目地址：https://github.com/Robbyant/lingbot-depth）。每个样本均包含RGB图像、传感器原始深度图与真值深度图。

<p align="center">
<img src="teaser.png" width="100%">
</p>

## 子数据集

| 名称 | 描述 | 样本数量 |
|------|-------------|--------:|
| **RobbyReal** | 由多台RGB-D相机采集的真实世界室内场景数据 | 1,400,000 |
| **RobbyVla** | VLA（视觉-语言-动作，Vision-Language-Action）机器人操控任务中采集的真实世界数据 | 580,960 |
| **RobbySim** | 从两个相机视角渲染得到的仿真数据 | 999,264 |
| **RobbySimVal** | 仿真数据的验证划分集 | 38,976 |
| **总计** | | **3,019,200** |

## 目录结构

### RobbyReal

RobbyReal/
└── <场景ID>/ # 例如 0001_01_room
├── orbbec_335_<序列ID>/
│ ├── color/ # RGB图像目录
│ ├── gtdepth/ # 真值深度图目录
│ ├── rawdepth/ # 原始传感器深度图目录
│ └── intrinsic.txt # 相机内参文件
├── orbbec_335L_<序列ID>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
├── realsense_D415_<序列ID>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
├── realsense_D435_<序列ID>/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── realsense_D455_<序列ID>/
├── color/
├── gtdepth/
├── rawdepth/
└── intrinsic.txt

### RobbyVla

在VLA（视觉-语言-动作）机器人操控任务中，于两个机器人平台上采集得到的数据。

RobbyVla/
├── franka/ # Franka机械臂
│ └── <序列ID>/
│ ├── left_realsense405/
│ │ ├── color/
│ │ ├── gtdepth/
│ │ ├── rawdepth/
│ │ └── intrinsic.txt
│ └── right_realsense405/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── ur7e/ # UR7e机械臂
└── <序列ID>/
├── left_realsense405/
│ ├── color/
│ ├── gtdepth/
│ ├── rawdepth/
│ └── intrinsic.txt
└── right_realsense405/
├── color/
├── gtdepth/
├── rawdepth/
└── intrinsic.txt

### RobbySim

从`object_view`（物体视角）与`rrt_view`（快速扩展随机树RRT视角）两个相机视角渲染得到的仿真数据。

RobbySim/
├── intrinsics.txt # 相机内参文件
├── object_view/
│ └── <场景ID>/<相机ID>/ # 例如 xxxx/00、xxxx/01
│ ├── 0001_cam0_left.jpg # RGB图像
│ ├── 0001_cam0_depth.png # 真值深度图
│ └── 0001_cam0_rmd2c.png # 原始深度图
└── rrt_view/
└── <场景ID>/<相机ID>/
├── 0001_cam0_left.jpg
├── 0001_cam0_depth.png
└── 0001_cam0_rmd2c.png

### RobbySimVal

仿真数据的验证划分集。

RobbySimVal/
└── val_view/
└── <场景ID>/<相机ID>/ # 例如 xxxx/00、xxxx/01
├── intrinsics.txt # 相机内参文件
├── 0001_cam0_rgb.left.jpg # RGB图像
├── 0001_cam0_depth_left.png # 真值深度图
└── 0001_cam0_rawdepth.left.png # 原始深度图

## 文件说明

| 文件/文件夹 | 说明 |
|-------------|-------------|
| `color/` / `*_rgb*.jpg` | RGB图像 |
| `gtdepth/` / `*_depth*.png` | 真值深度图（16位PNG格式，单位：毫米） |
| `rawdepth/` / `*_rawdepth*.png` / `*_rmd2c*.png` | 后处理前的传感器原始深度图（16位PNG格式，单位：毫米） |
| `intrinsic.txt` / `intrinsics.txt` | 相机内参参数文件 |