10Kh-RealOmin-OpenData

Boasting over 10,000 hours of cumulative data and 1 million+ clips, it ranks as the largest open-source embodied intelligence dataset in the industry.
# Compared with other datasets, it has the following advantages:
1. Ample Data Volume & Strong Generalization
Each skill is supported by sufficient data, collected from over 3,000 households and nearly 10,000 distinct fine-grained targets. It avoids simple repetitions and ensures robust generalization.
2. Authentic Scenarios & Focused Skills
Captured from natural operations in real households, we avoid skill fragmentation that compromises quality. Instead, we focus on 10 key household scenarios and 30 core skills.
3. Bimanual & Long-duration Tasks
Full recordings of the entire process of complex household chores and cleaning. Data collection by GenDAS Gripper.
4. Multi-modal & High-quality Data
Includes large-FOV raw images, trajectories, annotations and joint movements. Trajectory reconstruction ensures industry-leading precision and quality.
# Dataset Statistics
| Attribute | Value |
|-------------------------|----------------------|
| Median Clip Length | 210.0 seconds |
| Storage Size | 95 TB |
| Format | mcap |
| Resolution | 1600*1296 |
| Frame Rate | 30 fps |
| Camera Type | Large FOV Fisheye Camera |
| IMU | Yes 6-axis |
| Tactile Array Spatial | Yes |
| Array Spatial Resolution | 1 mm |
| Device |Gen DAS Gripper |

# Stage 1 Content：
**We have uploaded the data of Stage 1. This is only a small fraction and we will complete updates for the remaining skills as soon as possible.**
Stage 1 covers 12 skills across 4 major scenario tasks .Total duration: 950 hours, clips: 39,761, storage 3.45TB.
| Task | Skill |
|-------------------------------|--------------------------------|
| Folding_Clothes_and_Zipper_Operations | fold_and_store_clothes |
| | zip_clothes |
| Cooking_and_Kitchen_Clean | clean_container |
| | unscrew_bottle_cap_and_pour |
| | clean_bowl |
| Organize_Clutter | desktop_object_sorting |
| | fold_towel |
| | fold_and_store_shopping_bag |
| | drawer_to_take_items |
| | drawer_to_place_items |
| Shoes_Handling | lace_up_shoes_with_both_hands |
| | organize_scattered_shoes |

And synchronize the progress across major social platforms.In addition to the data, we will also provide relevant support including format conversion and usage guidance，here is the link GitHub - genrobot-ai/das-datakit.

# Contact Us
Any questions, suggestions or desired data collection scenarios/skills are welcome during usage. Let’s co-build this project to digitize all human skills.
X：https://x.com/GenrobotAI
Linkin：https://www.linkedin.com/company/108767412/admin/dashboard/
Email：[e-mail]

# Dataset Structure
```
The mcap files are stored in the final leaf folders of the file directory structure.Note: Each mcap file represents one piece of task data.
```

# Data Format
Dual-arm tasks: robot0 and robot1 represent the left and right grippers respectively. Each gripper contains the following topics:
``` python
/robot0/sensor/camera0/compressed # Fisheye camera image data — compressed and encoded in H.264 format.
/robot0/sensor/camera0/camera_info # Fisheye Intrinsic and Extrinsic Parameters
/robot0/sensor/imu # Inertial Measurement Unit (IMU) Data
/robot0/sensor/magnetic_encoder # Magnetic encoder data: gripper opening distance
/robot0/vio/eef_pose # Trajectory data
```
Topics are serialized using Protobuf for persistent storage

/robot0/sensor/camera0/compressed:
``` protobuf
// A compressed image
message CompressedImage {
// Timestamp of image
google.protobuf.Timestamp timestamp = 1;

// frame id
string frame_id = 4;

// Compressed image data, h264 video stream
bytes data = 2;

// Image format
// Supported values: `webp`, `jpeg`, `png`, `h264`
string format = 3;

// common header, timestamp is inside it
Header header = 8;
}

message Header {
string module_name = 1;
uint32 sequence_num = 2;
uint64 timestamp = 3;
string topic_name = 4;
double expect_hz = 5;
repeated Input inputs = 6;
}
```
/robot0/sensor/camera0/camera_info:
``` protobuf
// Camera calibration parameters
message CameraCalibration {
// not used
google.protobuf.Timestamp timestamp = 1;

// frame id
string frame_id = 9;

// Image width
fixed32 width = 2;

// Image height
fixed32 height = 3;

// Name of distortion model
string distortion_model = 4;

// Distortion parameters
repeated double D = 5;

// Intrinsic camera matrix (3x3 row-major matrix)
//
// A 3x3 row-major matrix for the raw (distorted) image.
//
// Projects 3D points in the camera coordinate frame to 2D pixel coordinates using the focal lengths (fx, fy) and principal point (cx, cy).
//
// ```
// [fx 0 cx]
// K = [ 0 fy cy]
// [ 0 0 1]
// ```
repeated double K = 6; // length 9

// Rectification matrix (stereo cameras only, 3x3 row-major matrix)
//
// A rotation matrix aligning the camera coordinate system to the ideal stereo image plane so that epipolar lines in both stereo images are parallel.
repeated double R = 7; // length 9

// Projection/camera matrix (stereo cameras only, 3x4 row-major matrix)
// [fx' 0 cx' Tx]
// P = [ 0 fy' cy' Ty]
// [ 0 0 1 0]
repeated double P = 8; // length 12

// transform from camera to base frame
repeated double T_b_c = 10; // length 7, [tx ty tz qx qy qz qw]
// common header
Header header = 11;
}
```
/robot0/sensor/imu:
``` protobuf
// IMU message
message IMUMeasurement {
// common header
arnold.common.proto.Header header = 1;
// frame id
string frame_id = 2;
foxglove.Vector3 angular_velocity = 3;
// Acceleration data in g-force units
foxglove.Vector3 linear_acceleration = 4;
// float temperature = 5;
// repeated float angular_velocity_covariance = 6;
// repeated float linear_acceleration_covariance = 7;
}
```
/robot0/sensor/magnetic_encoder:
``` protobuf
message MagneticEncoderMeasurement {
// common header
arnold.common.proto.Header header = 1;
// frame id
string frame_id = 2;
// Distance between gripper fingers, 0-0.103m, 0 means closed
double value = 3;
}
```
/robot0/vio/eef_pose:
``` protobuf
// A timestamped pose for an object or reference frame in 3D space
message PoseInFrame {
// not used
google.protobuf.Timestamp timestamp = 1;

// Frame id
string frame_id = 2;

// Pose in 3D space
foxglove.Pose pose = 3;
// linear vel
foxglove.Vector3 linear_vel= 4;
// angular_vel
foxglove.Vector3 angular_vel = 5;
// common header
arnold.common.proto.Header header = 6;
}
```

# How to Vis Data
web view tool；
```
https://monitor.genrobot.click/#/index
```

# How to Load Data
reference：
```
GitHub - genrobot-ai/das-datakit

本数据集累计数据时长超10000小时，包含百万余条片段，是业内规模最大的开源具身智能（embodied intelligence）数据集。
# 相较于其他同类数据集，本数据集具备以下优势：
1. 数据体量充足，泛化能力强劲
每项技能均配有足量数据，采集自3000余户家庭与近10000个差异化细粒度目标，避免了简单重复，确保模型具备优秀的泛化性能。
2. 场景真实，技能聚焦
数据采集自真实家庭中的自然操作场景，规避了破坏数据质量的技能碎片化问题，本次数据集聚焦10大核心家居场景与30项关键技能。
3. 支持双手操作与长时任务
完整记录复杂家居劳作与清洁流程的全过程，数据由GenDAS机械手（GenDAS Gripper）采集。
4. 多模态且高质量
包含大视场原始图像、轨迹、标注与关节运动数据。通过轨迹重构技术，实现了行业领先的精度与数据质量。
# 数据集统计信息
| 属性 | 数值 |
|-------------------------|----------------------|
| 片段中位时长 | 210.0 秒 |
| 存储容量 | 95 TB |
| 存储格式 | mcap |
| 分辨率 | 1600*1296 |
| 帧率 | 30 fps |
| 相机类型 | 大视场鱼眼相机（Large FOV Fisheye Camera） |
| 惯性测量单元（IMU） | 支持，6轴 |
| 触觉阵列传感器 | 支持 |
| 触觉阵列空间分辨率 | 1 毫米 |
| 采集设备 | Gen DAS机械手（Gen DAS Gripper） |

# 第一阶段内容
**我们已上传第一阶段数据集。本阶段数据仅为全部数据的一小部分，我们将尽快完成剩余技能相关数据的更新工作。**
第一阶段涵盖4大核心场景任务下的12项技能，总时长950小时，包含39761条数据片段，存储占用3.45 TB。
| 任务分类 | 技能名称 |
|-------------------------------|--------------------------------|
| 衣物折叠与拉链操作 | 折叠并收纳衣物 |
| | 衣物拉链操作 |
| 烹饪与厨房清洁 | 清洁容器 |
| | 拧开瓶盖并倾倒 |
| | 清洁碗具 |
| 杂物整理 | 桌面物品整理 |
| | 折叠毛巾 |
| | 折叠并收纳购物袋 |
| | 从抽屉取出物品 |
| | 将物品放入抽屉 |
| 鞋具处理 | 双手系鞋带 |
| | 整理散落的鞋具 |

我们将同步在各大社交平台更新项目进展。除数据集外，我们还将提供格式转换、使用指南等相关支持，相关链接：GitHub - genrobot-ai/das-datakit。

# 联系我们
使用过程中如有任何问题、建议或希望采集的场景/技能需求，欢迎随时与我们联系。让我们携手共建该项目，推动全人类技能的数字化转型。
X平台：https://x.com/GenrobotAI
领英（LinkedIn）：https://www.linkedin.com/company/108767412/admin/dashboard/
电子邮箱：[e-mail]

# 数据集结构

mcap格式文件存储于文件目录结构的最底层叶文件夹中。注意：每个mcap文件对应一项任务数据。

# 数据格式
双臂任务场景下：robot0与robot1分别对应左、右机械手。每个机械手包含以下话题（Topic）：
python
/robot0/sensor/camera0/compressed # 鱼眼相机图像数据——采用H.264格式压缩编码
/robot0/sensor/camera0/camera_info # 鱼眼相机内外参信息
/robot0/sensor/imu # 惯性测量单元（IMU）数据
/robot0/sensor/magnetic_encoder # 磁编码器数据：机械手开合距离
/robot0/vio/eef_pose # 末端执行器轨迹数据

话题数据采用Protobuf进行序列化以实现持久化存储。

/robot0/sensor/camera0/compressed:
protobuf
// 压缩图像消息
message CompressedImage {
// 图像时间戳
google.protobuf.Timestamp timestamp = 1;

// 帧ID
string frame_id = 4;

// 压缩图像数据，H.264视频流
bytes data = 2;

// 图像格式
// 支持格式：`webp`、`jpeg`、`png`、`h264`
string format = 3;

// 通用消息头，内部包含时间戳
Header header = 8;
}

message Header {
string module_name = 1;
uint32 sequence_num = 2;
uint64 timestamp = 3;
string topic_name = 4;
double expect_hz = 5;
repeated Input inputs = 6;
}

/robot0/sensor/camera0/camera_info:
protobuf
// 相机标定参数消息
message CameraCalibration {
// 未使用字段
google.protobuf.Timestamp timestamp = 1;

// 帧ID
string frame_id = 9;

// 图像宽度
fixed32 width = 2;

// 图像高度
fixed32 height = 3;

// 畸变模型名称
string distortion_model = 4;

// 畸变参数
repeated double D = 5;

// 相机内参矩阵（3x3行优先矩阵）
// 将相机坐标系下的3D点投影至2D像素坐标，使用焦距(fx, fy)与主点(cx, cy)
//
// [fx 0 cx]
// K = [ 0 fy cy]
// [ 0 0 1]
//
repeated double K = 6; // 长度为9

// 校正矩阵（仅立体相机使用，3x3行优先矩阵）
// 用于将相机坐标系旋转至理想立体成像平面，使两幅立体图像的极线平行
repeated double R = 7; // 长度为9

// 投影/相机矩阵（仅立体相机使用，3x4行优先矩阵）
// [fx' 0 cx' Tx]
// P = [ 0 fy' cy' Ty]
// [ 0 0 1 0]
repeated double P = 8; // 长度为12

// 相机到基坐标系的变换矩阵
repeated double T_b_c = 10; // 长度为7，格式为[tx ty tz qx qy qz qw]
// 通用消息头
Header header = 11;
}

/robot0/sensor/imu:
protobuf
// IMU测量消息
message IMUMeasurement {
// 通用消息头
arnold.common.proto.Header header = 1;
// 帧ID
string frame_id = 2;
foxglove.Vector3 angular_velocity = 3;
// 以g为单位的加速度数据
foxglove.Vector3 linear_acceleration = 4;
// float temperature = 5;
// repeated float angular_velocity_covariance = 6;
// repeated float linear_acceleration_covariance = 7;
}

/robot0/sensor/magnetic_encoder:
protobuf
message MagneticEncoderMeasurement {
// 通用消息头
arnold.common.proto.Header header = 1;
// 帧ID
string frame_id = 2;
// 机械手手指开合距离，范围0-0.103m，0表示完全闭合
double value = 3;
}

/robot0/vio/eef_pose:
protobuf
// 三维空间中物体或参考帧的带时间戳位姿消息
message PoseInFrame {
// 未使用字段
google.protobuf.Timestamp timestamp = 1;

// 帧ID
string frame_id = 2;

// 三维空间位姿
foxglove.Pose pose = 3;
// 线速度
foxglove.Vector3 linear_vel= 4;
// 角速度
foxglove.Vector3 angular_vel = 5;
// 通用消息头
arnold.common.proto.Header header = 6;
}

# 数据可视化方法
可视化工具：https://monitor.genrobot.click/#/index

# 数据加载方法
参考实现：GitHub - genrobot-ai/das-datakit

搜集汇总

背景与挑战

背景概述

该数据集是行业最大的开源具身智能数据集，累计超过1万小时数据和100万+片段，具备强大的泛化能力和真实家庭场景采集。它专注于10个关键场景和30个核心技能，提供多模态高质量数据，包括双手机器人任务和高精度传感器信息。第一阶段已上传部分内容，涵盖12项技能，数据格式为mcap，支持广泛的应用和研究。