Cobot_Magic_make_hamburger

# Cobot_Magic_make_hamburger

## 📋 Overview

This dataset uses an extended format based on LeRobot and is fully compatible with LeRobot.

**Robot Type:** `Cobot_Magic`
| **Codebase Version:** `v2.1`
**End-Effector Type:** `two_finger_gripper`

## 🏠 Scene Types

This dataset covers the following scene types:
- `restaurant`

## 🤖 Atomic Actions

This dataset includes the following atomic actions:
- `grasp`
- `place`
- `pick`

## 📊 Dataset Statistics

| Metric | Value |
|--------|-------|
| **Total Episodes** | 3077 |
| **Total Frames** | 1596406 |
| **Total Tasks** | 4 |
| **Total Videos** | 12308 |
| **Total Chunks** | 4 |
| **Chunk Size** | 1000 |
| **FPS** | 30 |
| **Dataset Size** | 46.4GB |

## 👥 Authors

### Contributors
This dataset is contributed by:
- [RoboCOIN](https://flagopen.github.io/RoboCOIN/) - RoboCOIN Team

## 🔗 Links

- **🏠 Homepage:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **📄 Paper:** [https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441)
- **💻 Repository:** [https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN)
- **🌐 Project Page:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **🐛 Issues:** [https://github.com/FlagOpen/RoboCOIN/issues](https://github.com/FlagOpen/RoboCOIN/issues)
- **📜 License:** apache-2.0

## 🏷️ Dataset Tags

- `RoboCOIN`
- `LeRobot`

## 🎯 Task Descriptions

### Primary Tasks
make a hamburger with cheese.
make a hamburger with lettuce.
make a hamburger with cheese and lettuce.
place the sliced bread, cheese, lettuce, meat patties, and tomato slices on a plate.

### Sub-Tasks
This dataset includes 48 distinct subtasks:

1. **Abnormal**
2. **Discard.**
3. **End**
4. **Grab the lettuce leaf with the right hand.**
5. **Grab the patty with the right hand.**
6. **null**
7. **Pick up the bottom bread slice with the left hand.**
8. **Pick up the bottom bread with the left hand.**
9. **Pick up the bread slice with the left gripper**
10. **Pick up the bread slices and place them on the tray with the left gripper**
11. **Pick up the cheese slice with the left gripper**
12. **Pick up the cheese slice with the left hand.**
13. **Pick up the cheese slice with the right gripper**
14. **Pick up the cutlet with the right gripper**
15. **Pick up the hamburger lid with the left gripper**
16. **Pick up the hamburger lid with the right gripper**
17. **Pick up the lettuce leaf with the right gripper**
18. **Pick up the meat patty and place it on top of the tomato with the right gripper**
19. **Pick up the purple cabbage with the left gripper**
20. **Pick up the tomato slice with the left gripper**
21. **Pick up the tomato slice with the left hand.**
22. **Pick up the tomato slice with the right gripper**
23. **Pick up the top bread slice with the right hand.**
24. **Place the bottom bread slice on the tray with the left hand.**
25. **Place the bread on the tray with the left hand.**
26. **Place the bread slice on the tray with the left gripper**
27. **Place the cheese slice on the cutlet with the left gripper**
28. **Place the cheese slice on the cutlet with the right gripper**
29. **Place the cheese slice on the patty with the left hand.**
30. **Place the cutlet on the lettuce leaf with the right gripper**
31. **Place the cutlet on the tomato slice with the right gripper**
32. **Place the hamburger lid on the cutlet with the right gripper**
33. **Place the hamburger lid on the cheese slice with the left gripper**
34. **Place the hamburger lid on the cheese slice with the right gripper**
35. **Place the hamburger lid on the cutlet slice with the left gripper**
36. **Place the hamburger lid on the purple cabbage with the right gripper**
37. **Place the lettuce leaf on the bread slice with the right gripper**
38. **Place the lettuce leaf on the bottom bread with the right hand.**
39. **Place the lettuce leaf on the bread slice with the right hand.**
40. **Place the patty on the tomato slice with the right hand.**
41. **Place the purple cabbage on the cutlet with the left gripper**
42. **Place the purple cabbage on the tomato slice with the left gripper**
43. **Place the tomato slice on the bread slice with the left gripper**
44. **Place the tomato slice on the bread slice with the right gripper**
45. **Place the tomato slice on the lettuce leaf with the left gripper**
46. **Place the tomato slice on the lettuce leaf with the left hand.**
47. **Place the tomato slice on the lettuce leaf with the right gripper**
48. **Place the top bread slice on the cheese with the right hand.**

## 🎥 Camera Views

This dataset includes 4 camera views.

## 🏷️ Available Annotations

This dataset includes rich annotations to support diverse learning approaches:

### Subtask Annotations
- **Subtask Segmentation**: Fine-grained subtask segmentation and labeling
### Scene Annotations
- **Scene-level Descriptions**: Semantic scene classifications and descriptions
### End-Effector Annotations
- **Direction**: Movement direction classifications for robot end-effectors
- **Velocity**: Velocity magnitude categorizations during manipulation
- **Acceleration**: Acceleration magnitude classifications for motion analysis

### Gripper Annotations
- **Gripper Mode**: Open/close state annotations for gripper control
- **Gripper Activity**: Activity state classifications (active/inactive)

### Additional Features
- **End-Effector Simulation Pose**: 6D pose information for end-effectors in simulation space
- Available for both state and action
- **Gripper Opening Scale**: Continuous gripper opening measurements
- Available for both state and action

## 📂 Data Splits

The dataset is organized into the following splits:

- **Training**: Episodes 0:3076

## 📁 Dataset Structure

This dataset follows the LeRobot format and contains the following components:

### Data Files
- **Videos**: Compressed video files containing RGB camera observations
- **State Data**: Robot joint positions, velocities, and other state information
- **Action Data**: Robot action commands and trajectories
- **Metadata**: Episode metadata, timestamps, and annotations

### File Organization
- **Data Path Pattern**: `data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet`
- **Video Path Pattern**: `videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4`
- **Chunking**: Data is organized into 4 chunk(s)
of size 1000

### Features Schema

The dataset includes the following features:

#### Visual Observations
- **observation.images.cam_high_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_high_realsense_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_left_wrist_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_right_wrist_rgb**: video
- FPS: 30
- Codec: av1

#### State and Action- **observation.state**: float32- **action**: float32

#### Temporal Information
- **timestamp**: float32
- **frame_index**: int64
- **episode_index**: int64
- **index**: int64
- **task_index**: int64

#### Annotations
- **subtask_annotation**: int32
- **scene_annotation**: int32

#### Motion Features
- **eef_sim_pose_state**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_sim_pose_action**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_direction_state**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_direction_action**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_velocity_state**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_velocity_action**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_acc_mag_state**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag
- **eef_acc_mag_action**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag

#### Gripper Features
- **gripper_open_scale_state**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_open_scale_action**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_mode_state**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_mode_action**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_activity_state**: int32
- Dimensions: left_gripper_activity, right_gripper_activity

### Meta Information

The complete dataset metadata is available in [meta/info.json](meta/info.json):

```json
{"codebase_version": "v2.1", "robot_type": "agilex_cobot_decoupled_magic", "total_episodes": 3077, "total_frames": 1596406, "total_tasks": 4, "total_videos": 12308, "total_chunks": 4, "chunks_size": 1000, "fps": 30, "splits": {"train": "0:3076"}, "data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet", "video_path": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4", "features": {"observation.images.cam_high_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_high_realsense_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_left_wrist_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_right_wrist_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.state": {"dtype": "float32", "shape": [26], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "left_eef_pos_x_m", "left_eef_pos_y_m", "left_eef_pos_z_m", "left_eef_rot_euler_x_rad", "left_eef_rot_euler_y_rad", "left_eef_rot_euler_z_rad", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open", "right_eef_pos_x_m", "right_eef_pos_y_m", "right_eef_pos_z_m", "right_eef_rot_euler_x_rad", "right_eef_rot_euler_y_rad", "right_eef_rot_euler_z_rad"]}, "action": {"dtype": "float32", "shape": [26], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "left_eef_pos_x_m", "left_eef_pos_y_m", "left_eef_pos_z_m", "left_eef_rot_euler_x_rad", "left_eef_rot_euler_y_rad", "left_eef_rot_euler_z_rad", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open", "right_eef_pos_x_m", "right_eef_pos_y_m", "right_eef_pos_z_m", "right_eef_rot_euler_x_rad", "right_eef_rot_euler_y_rad", "right_eef_rot_euler_z_rad"]}, "timestamp": {"dtype": "float32", "shape": [1], "names": null}, "frame_index": {"dtype": "int64", "shape": [1], "names": null}, "episode_index": {"dtype": "int64", "shape": [1], "names": null}, "index": {"dtype": "int64", "shape": [1], "names": null}, "task_index": {"dtype": "int64", "shape": [1], "names": null}, "subtask_annotation": {"names": null, "dtype": "int32", "shape": [5]}, "scene_annotation": {"names": null, "dtype": "int32", "shape": [1]}, "eef_sim_pose_state": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_sim_pose_action": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_direction_state": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_direction_action": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_velocity_state": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_velocity_action": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_state": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_action": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "gripper_open_scale_state": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_open_scale_action": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_mode_state": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_mode_action": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_activity_state": {"names": ["left_gripper_activity", "right_gripper_activity"], "dtype": "int32", "shape": [2]}}}
```

### Directory Structure

The dataset is organized as follows (showing leaf directories with first 5 files only):

```
Cobot_Magic_make_hamburger_qced_hardlink/
├── annotations/
│ ├── eef_acc_mag_annotation.jsonl
│ ├── eef_direction_annotation.jsonl
│ ├── eef_velocity_annotation.jsonl
│ ├── gripper_activity_annotation.jsonl
│ ├── gripper_mode_annotation.jsonl
│ └── (...)
├── data/
│ ├── chunk-000/
│ │ ├── episode_000000.parquet
│ │ ├── episode_000001.parquet
│ │ ├── episode_000002.parquet
│ │ ├── episode_000003.parquet
│ │ ├── episode_000004.parquet
│ │ └── (...)
│ ├── chunk-001/
│ │ ├── episode_001000.parquet
│ │ ├── episode_001001.parquet
│ │ ├── episode_001002.parquet
│ │ ├── episode_001003.parquet
│ │ ├── episode_001004.parquet
│ │ └── (...)
│ ├── chunk-002/
│ │ ├── episode_002000.parquet
│ │ ├── episode_002001.parquet
│ │ ├── episode_002002.parquet
│ │ ├── episode_002003.parquet
│ │ ├── episode_002004.parquet
│ │ └── (...)
│ └── chunk-003/
│ ├── episode_003000.parquet
│ ├── episode_003001.parquet
│ ├── episode_003002.parquet
│ ├── episode_003003.parquet
│ ├── episode_003004.parquet
│ └── (...)
├── meta/
│ ├── episodes.jsonl
│ ├── episodes_stats.jsonl
│ ├── info.json
│ └── tasks.jsonl
└── videos/
├── chunk-000/
│ ├── observation.images.cam_high_realsense_rgb/
│ │ ├── episode_000000.mp4
│ │ ├── episode_000001.mp4
│ │ ├── episode_000002.mp4
│ │ ├── episode_000003.mp4
│ │ ├── episode_000004.mp4
│ │ └── (...)
│ ├── observation.images.cam_high_rgb/
│ │ ├── episode_000000.mp4
│ │ ├── episode_000001.mp4
│ │ ├── episode_000002.mp4
│ │ ├── episode_000003.mp4
│ │ ├── episode_000004.mp4
│ │ └── (...)
│ ├── observation.images.cam_left_wrist_rgb/
│ │ ├── episode_000000.mp4
│ │ ├── episode_000001.mp4
│ │ ├── episode_000002.mp4
│ │ ├── episode_000003.mp4
│ │ ├── episode_000004.mp4
│ │ └── (...)
│ └── observation.images.cam_right_wrist_rgb/
│ ├── episode_000000.mp4
│ ├── episode_000001.mp4
│ ├── episode_000002.mp4
│ ├── episode_000003.mp4
│ ├── episode_000004.mp4
│ └── (...)
├── chunk-001/
│ ├── observation.images.cam_high_realsense_rgb/
│ │ ├── episode_001000.mp4
│ │ ├── episode_001001.mp4
│ │ ├── episode_001002.mp4
│ │ ├── episode_001003.mp4
│ │ ├── episode_001004.mp4
│ │ └── (...)
│ ├── observation.images.cam_high_rgb/
│ │ ├── episode_001000.mp4
│ │ ├── episode_001001.mp4
│ │ ├── episode_001002.mp4
│ │ ├── episode_001003.mp4
│ │ ├── episode_001004.mp4
│ │ └── (...)
│ ├── observation.images.cam_left_wrist_rgb/
│ │ ├── episode_001000.mp4
│ │ ├── episode_001001.mp4
│ │ ├── episode_001002.mp4
│ │ ├── episode_001003.mp4
│ │ ├── episode_001004.mp4
│ │ └── (...)
│ └── observation.images.cam_right_wrist_rgb/
│ ├── episode_001000.mp4
│ ├── episode_001001.mp4
│ ├── episode_001002.mp4
│ ├── episode_001003.mp4
│ ├── episode_001004.mp4
│ └── (...)
├── chunk-002/
│ ├── observation.images.cam_high_realsense_rgb/
│ │ ├── episode_002000.mp4
│ │ ├── episode_002001.mp4
│ │ ├── episode_002002.mp4
│ │ ├── episode_002003.mp4
│ │ ├── episode_002004.mp4
│ │ └── (...)
│ ├── observation.images.cam_high_rgb/
│ │ ├── episode_002000.mp4
│ │ ├── episode_002001.mp4
│ │ ├── episode_002002.mp4
│ │ ├── episode_002003.mp4
│ │ ├── episode_002004.mp4
│ │ └── (...)
│ ├── observation.images.cam_left_wrist_rgb/
│ │ ├── episode_002000.mp4
│ │ ├── episode_002001.mp4
│ │ ├── episode_002002.mp4
│ │ ├── episode_002003.mp4
│ │ ├── episode_002004.mp4
│ │ └── (...)
│ └── observation.images.cam_right_wrist_rgb/
│ ├── episode_002000.mp4
│ ├── episode_002001.mp4
│ ├── episode_002002.mp4
│ ├── episode_002003.mp4
│ ├── episode_002004.mp4
│ └── (...)
└── chunk-003/
├── observation.images.cam_high_realsense_rgb/
│ ├── episode_003000.mp4
│ ├── episode_003001.mp4
│ ├── episode_003002.mp4
│ ├── episode_003003.mp4
│ ├── episode_003004.mp4
│ └── (...)
├── observation.images.cam_high_rgb/
│ ├── episode_003000.mp4
│ ├── episode_003001.mp4
│ ├── episode_003002.mp4
│ ├── episode_003003.mp4
│ ├── episode_003004.mp4
│ └── (...)
├── observation.images.cam_left_wrist_rgb/
│ ├── episode_003000.mp4
│ ├── episode_003001.mp4
│ ├── episode_003002.mp4
│ ├── episode_003003.mp4
│ ├── episode_003004.mp4
│ └── (...)
└── observation.images.cam_right_wrist_rgb/
├── episode_003000.mp4
├── episode_003001.mp4
├── episode_003002.mp4
├── episode_003003.mp4
├── episode_003004.mp4
└── (...)
```

## 📞 Contact and Support

For questions, issues, or feedback regarding this dataset, please contact:
- **Email:** None
For questions, issues, or feedback regarding this dataset, please contact us.
### Support
For technical support, please open an issue on our GitHub repository.

## 📄 License

This dataset is released under the **apache-2.0** license.

Please refer to the LICENSE file for full license terms and conditions.

## 📚 Citation

If you use this dataset in your research, please cite:

```bibtex
@article{robocoin,
title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
journal={arXiv preprint arXiv:2511.17441},
url = {https://arxiv.org/abs/2511.17441},
year={2025}
}
```

### Additional References

If you use this dataset, please also consider citing:
- LeRobot Framework: https://github.com/huggingface/lerobot

## 📌 Version Information

## Version History
- v1.0.0 (2025-11): Initial release