Cobot_Magic_desktop_organization

# Cobot_Magic_desktop_organization

## 📋 Overview

This dataset uses an extended format based on LeRobot and is fully compatible with LeRobot.

**Robot Type:** `Cobot_Magic`
| **Codebase Version:** `v2.1`
**End-Effector Type:** `two_finger_gripper`

## 🏠 Scene Types

This dataset covers the following scene types:
- `home`
- `office`

## 🤖 Atomic Actions

This dataset includes the following atomic actions:
- `grasp`
- `pick`
- `place`

## 📊 Dataset Statistics

| Metric | Value |
|--------|-------|
| **Total Episodes** | 1288 |
| **Total Frames** | 2064431 |
| **Total Tasks** | 6 |
| **Total Videos** | 3864 |
| **Total Chunks** | 2 |
| **Chunk Size** | 1000 |
| **FPS** | 30 |
| **Dataset Size** | 31.3GB |

## 👥 Authors

### Contributors
This dataset is contributed by:
- [RoboCOIN](https://flagopen.github.io/RoboCOIN/) - RoboCOIN Team

## 🔗 Links

- **🏠 Homepage:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **📄 Paper:** [https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441)
- **💻 Repository:** [https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN)
- **🌐 Project Page:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **🐛 Issues:** [https://github.com/FlagOpen/RoboCOIN/issues](https://github.com/FlagOpen/RoboCOIN/issues)
- **📜 License:** apache-2.0

## 🏷️ Dataset Tags

- `RoboCOIN`
- `LeRobot`

## 🎯 Task Descriptions

### Primary Tasks
put the mango on the plate, put the waste paper in the basket, place the water bottle upright near the plate, put the eraser in the pen holder, put the pen in the pen holder.
put the bread on the plate, put the small pieces of paper in the basket, place thecoffee near the plate, put the pencil sharpener in the pen holder, put the rulerin the pen holder, put the pen in the pen holder.
put the pear on the plate, put the waste paper in the basket, place the water bottleupright near the plate, put the knife in the pen holder, put the ruler in the penholder, put the pen in the pen holder.
put the grape on a plate, put the plastic sheet in the basket, place the teacupneatly near the plate, put the scissor in the pen holder, put the ruler in the penholder, put the colored glue in the pen holder.
put the mango and banana on the fruit tray, put the paper ball in the garbage bin,straighten the water bottle, put the eraser and pen in the pen holder.
pick up the green basket, put the cola inside the basket, and then put the basketdown.

### Sub-Tasks
This dataset includes 108 distinct subtasks:

1. **abnormal**
2. **End**
3. **Grasp the banana with left gripper**
4. **Grasp the basket with left gripper**
5. **Grasp the black pen with left gripper**
6. **Grasp the black pen with right gripper**
7. **Grasp the blue pen with left gripper**
8. **Grasp the blue pen with right gripper**
9. **Grasp the bottle with left gripper**
10. **Grasp the cola can with left gripper**
11. **Grasp the cola can with right gripper**
12. **Grasp the eraser with left gripper**
13. **Grasp the eraser with right gripper**
14. **Grasp the gray pen with left gripper**
15. **Grasp the gray pen with right gripper**
16. **Grasp the mango with left gripper**
17. **Grasp the milk with left gripper**
18. **Grasp the milk with right gripper**
19. **Grasp the waste paper with left gripper**
20. **Grasp the waste paper with right gripper**
21. **Grasp the yellow pen with left gripper**
22. **Grasp the yellow pen with right gripper**
23. **Hand over the banana**
24. **Hand over the blue knife**
25. **Hand over the blue scissors**
26. **Hand over the bottle**
27. **Hand over the bread**
28. **Hand over the grapes**
29. **Hand over the pear**
30. **Hand over the pen**
31. **Hand over the pink scissors**
32. **Hand over the plastic**
33. **Hand over the red glue**
34. **Hand over the ruler**
35. **Hand over the silver glue**
36. **Hand over the waste paper**
37. **Hand over the yellow glue**
38. **Hand over the yellow knife**
39. **Hand the basket to the right gripper with the left gripper**
40. **Lift the basket with left gripper**
41. **null**
42. **Pick up the banana**
43. **pick up the blue knife**
44. **pick up the blue pencil sharpener**
45. **pick up the blue scissors**
46. **Pick up the bottle with the left arm**
47. **Pick up the bottle with the right arm**
48. **pick up the bread**
49. **Pick up the eraser**
50. **Pick up the grapes**
51. **Pick up the pear**
52. **Pick up the pen with left arm**
53. **Pick up the pen with right arm**
54. **pick up the pink scissors**
55. **Pick up the plastic**
56. **pick up the red glue**
57. **pick up the red pencil sharpener**
58. **pick up the ruler**
59. **pick up the silver glue**
60. **Pick up the waste paper**
61. **pick up the yellow glue**
62. **pick up the yellow knife**
63. **Place the banana in the plate**
64. **Place the banana in the plate with left gripper**
65. **Place the basket on the table with left gripper**
66. **Place the basket on the table with right gripper**
67. **Place the black pen in pen holder with left gripper**
68. **Place the black pen in pen holder with right gripper**
69. **Place the blue knife into the pen holder**
70. **Place the blue pen in pen holder with left gripper**
71. **Place the blue pen in pen holder with right gripper**
72. **place the blue pencil sharpener into the pen holder**
73. **Place the blue scissors into the pen holder**
74. **Place the bottle**
75. **place the bread in the plate**
76. **Place the cola can in the basket with left gripper**
77. **Place the cola can in the basket with right gripper**
78. **Place the cola can on the table with left gripper**
79. **Place the cola can on the table with right gripper**
80. **Place the eraser in pen holder with left gripper**
81. **Place the eraser in pen holder with right gripper**
82. **Place the eraser into the pen holder**
83. **Place the grapes in the plate**
84. **Place the gray pen in pen holder with left gripper**
85. **Place the gray pen in pen holder with right gripper**
86. **Place the mango in the plate with left gripper**
87. **Place the milk in the basket with left gripper**
88. **Place the milk in the basket with right gripper**
89. **Place the pear in the plate**
90. **Place the pen into the pen holder**
91. **Place the pink scissors into the pen holder**
92. **Place the plastic in the basket**
93. **place the red glue into the pen holder**
94. **Place the red pencil sharpener into the pen holder**
95. **Place the ruler into the pen holder**
96. **place the silver glue into the pen holder**
97. **Place the waste paper in the basket**
98. **Place the waste paper in the basket with left gripper**
99. **Place the waste paper in the basket with right gripper**
100. **place the yellow glue into the pen holder**
101. **place the yellow knife into the pen holder**
102. **Place the yellow pen in pen holder with left gripper**
103. **Place the yellow pen in pen holder with right gripper**
104. **Stand the bottle with left gripper**
105. **Static**
106. **transfer the bottle to the left arm**
107. **transfer the pen to the left arm**
108. **transfer the pen to the right arm**

## 🎥 Camera Views

This dataset includes 3 camera views.

## 🏷️ Available Annotations

This dataset includes rich annotations to support diverse learning approaches:

### Subtask Annotations
- **Subtask Segmentation**: Fine-grained subtask segmentation and labeling
### Scene Annotations
- **Scene-level Descriptions**: Semantic scene classifications and descriptions
### End-Effector Annotations
- **Direction**: Movement direction classifications for robot end-effectors
- **Velocity**: Velocity magnitude categorizations during manipulation
- **Acceleration**: Acceleration magnitude classifications for motion analysis

### Gripper Annotations
- **Gripper Mode**: Open/close state annotations for gripper control
- **Gripper Activity**: Activity state classifications (active/inactive)

### Additional Features
- **End-Effector Simulation Pose**: 6D pose information for end-effectors in simulation space
- Available for both state and action
- **Gripper Opening Scale**: Continuous gripper opening measurements
- Available for both state and action

## 📂 Data Splits

The dataset is organized into the following splits:

- **Training**: Episodes 0:1287

## 📁 Dataset Structure

This dataset follows the LeRobot format and contains the following components:

### Data Files
- **Videos**: Compressed video files containing RGB camera observations
- **State Data**: Robot joint positions, velocities, and other state information
- **Action Data**: Robot action commands and trajectories
- **Metadata**: Episode metadata, timestamps, and annotations

### File Organization
- **Data Path Pattern**: `data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet`
- **Video Path Pattern**: `videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4`
- **Chunking**: Data is organized into 2 chunk(s)
of size 1000

### Features Schema

The dataset includes the following features:

#### Visual Observations
- **observation.images.cam_high_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_left_wrist_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_right_wrist_rgb**: video
- FPS: 30
- Codec: av1

#### State and Action- **observation.state**: float32- **action**: float32

#### Temporal Information
- **timestamp**: float32
- **frame_index**: int64
- **episode_index**: int64
- **index**: int64
- **task_index**: int64

#### Annotations
- **subtask_annotation**: int32
- **scene_annotation**: int32

#### Motion Features
- **eef_sim_pose_state**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_sim_pose_action**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_direction_state**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_direction_action**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_velocity_state**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_velocity_action**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_acc_mag_state**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag
- **eef_acc_mag_action**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag

#### Gripper Features
- **gripper_open_scale_state**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_open_scale_action**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_mode_state**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_mode_action**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_activity_state**: int32
- Dimensions: left_gripper_activity, right_gripper_activity

### Meta Information

The complete dataset metadata is available in [meta/info.json](meta/info.json):

```json
{"codebase_version": "v2.1", "robot_type": "agilex_cobot_decoupled_magic", "total_episodes": 1288, "total_frames": 2064431, "total_tasks": 6, "total_videos": 3864, "total_chunks": 2, "chunks_size": 1000, "fps": 30, "splits": {"train": "0:1287"}, "data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet", "video_path": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4", "features": {"observation.images.cam_high_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_left_wrist_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_right_wrist_rgb": {"dtype": "video", "shape": [480, 640, 3], "names": ["height", "width", "channels"], "info": {"video.height": 480, "video.width": 640, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.state": {"dtype": "float32", "shape": [26], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "left_eef_pos_x_m", "left_eef_pos_y_m", "left_eef_pos_z_m", "left_eef_rot_euler_x_rad", "left_eef_rot_euler_y_rad", "left_eef_rot_euler_z_rad", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open", "right_eef_pos_x_m", "right_eef_pos_y_m", "right_eef_pos_z_m", "right_eef_rot_euler_x_rad", "right_eef_rot_euler_y_rad", "right_eef_rot_euler_z_rad"]}, "action": {"dtype": "float32", "shape": [26], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "left_eef_pos_x_m", "left_eef_pos_y_m", "left_eef_pos_z_m", "left_eef_rot_euler_x_rad", "left_eef_rot_euler_y_rad", "left_eef_rot_euler_z_rad", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open", "right_eef_pos_x_m", "right_eef_pos_y_m", "right_eef_pos_z_m", "right_eef_rot_euler_x_rad", "right_eef_rot_euler_y_rad", "right_eef_rot_euler_z_rad"]}, "timestamp": {"dtype": "float32", "shape": [1], "names": null}, "frame_index": {"dtype": "int64", "shape": [1], "names": null}, "episode_index": {"dtype": "int64", "shape": [1], "names": null}, "index": {"dtype": "int64", "shape": [1], "names": null}, "task_index": {"dtype": "int64", "shape": [1], "names": null}, "subtask_annotation": {"names": null, "dtype": "int32", "shape": [5]}, "scene_annotation": {"names": null, "dtype": "int32", "shape": [1]}, "eef_sim_pose_state": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_sim_pose_action": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_direction_state": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_direction_action": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_velocity_state": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_velocity_action": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_state": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_action": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "gripper_open_scale_state": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_open_scale_action": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_mode_state": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_mode_action": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_activity_state": {"names": ["left_gripper_activity", "right_gripper_activity"], "dtype": "int32", "shape": [2]}}}
```

### Directory Structure

The dataset is organized as follows (showing leaf directories with first 5 files only):

```
Cobot_Magic_desktop_organization_qced_hardlink/
├── annotations/
│ ├── eef_acc_mag_annotation.jsonl
│ ├── eef_direction_annotation.jsonl
│ ├── eef_velocity_annotation.jsonl
│ ├── gripper_activity_annotation.jsonl
│ ├── gripper_mode_annotation.jsonl
│ └── (...)
├── data/
│ ├── chunk-000/
│ │ ├── episode_000000.parquet
│ │ ├── episode_000001.parquet
│ │ ├── episode_000002.parquet
│ │ ├── episode_000003.parquet
│ │ ├── episode_000004.parquet
│ │ └── (...)
│ └── chunk-001/
│ ├── episode_001000.parquet
│ ├── episode_001001.parquet
│ ├── episode_001002.parquet
│ ├── episode_001003.parquet
│ ├── episode_001004.parquet
│ └── (...)
├── meta/
│ ├── episodes.jsonl
│ ├── episodes_stats.jsonl
│ ├── info.json
│ └── tasks.jsonl
└── videos/
├── chunk-000/
│ ├── observation.images.cam_high_rgb/
│ │ ├── episode_000000.mp4
│ │ ├── episode_000001.mp4
│ │ ├── episode_000002.mp4
│ │ ├── episode_000003.mp4
│ │ ├── episode_000004.mp4
│ │ └── (...)
│ ├── observation.images.cam_left_wrist_rgb/
│ │ ├── episode_000000.mp4
│ │ ├── episode_000001.mp4
│ │ ├── episode_000002.mp4
│ │ ├── episode_000003.mp4
│ │ ├── episode_000004.mp4
│ │ └── (...)
│ └── observation.images.cam_right_wrist_rgb/
│ ├── episode_000000.mp4
│ ├── episode_000001.mp4
│ ├── episode_000002.mp4
│ ├── episode_000003.mp4
│ ├── episode_000004.mp4
│ └── (...)
└── chunk-001/
├── observation.images.cam_high_rgb/
│ ├── episode_001000.mp4
│ ├── episode_001001.mp4
│ ├── episode_001002.mp4
│ ├── episode_001003.mp4
│ ├── episode_001004.mp4
│ └── (...)
├── observation.images.cam_left_wrist_rgb/
│ ├── episode_001000.mp4
│ ├── episode_001001.mp4
│ ├── episode_001002.mp4
│ ├── episode_001003.mp4
│ ├── episode_001004.mp4
│ └── (...)
└── observation.images.cam_right_wrist_rgb/
├── episode_001000.mp4
├── episode_001001.mp4
├── episode_001002.mp4
├── episode_001003.mp4
├── episode_001004.mp4
└── (...)
```

## 📞 Contact and Support

For questions, issues, or feedback regarding this dataset, please contact:
- **Email:** None
For questions, issues, or feedback regarding this dataset, please contact us.
### Support
For technical support, please open an issue on our GitHub repository.

## 📄 License

This dataset is released under the **apache-2.0** license.

Please refer to the LICENSE file for full license terms and conditions.

## 📚 Citation

If you use this dataset in your research, please cite:

```bibtex
@article{robocoin,
title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
journal={arXiv preprint arXiv:2511.17441},
url = {https://arxiv.org/abs/2511.17441},
year={2025}
}
```

### Additional References

If you use this dataset, please also consider citing:
- LeRobot Framework: https://github.com/huggingface/lerobot

## 📌 Version Information

## Version History
- v1.0.0 (2025-11): Initial release