Agilex_Split_Aloha_make_hamburger

# Agilex_Cobot_Magic_make_hamburger ## Dataset Description This dataset uses an extended format based on LeRobot and is fully compatible with LeRobot. ## Task Preview <video src="videos/chunk-000/observation.images.cam_high_realsense_rgb/episode_000000.mp4" controls width="640"></video> [View Video Directly](videos/chunk-000/observation.images.cam_high_realsense_rgb/episode_000000.mp4) ### Overview - **Total Episodes:** 3044 - **Total Frames:** 1591383 - **FPS:** 30 - **Dataset Size:** 46.71 GB - **Robot Name:** `Agilex_Cobot_Magic` - **End-Effector Type:** `two_finger_end_effector` - **Teleoperation Type:** `Due to some reasons, this dataset temporarily cannot provide the teleoperation type information.` - **Sensors:** `cam_high_rgb`, `cam_high_realsense_rgb`, `cam_left_wrist_rgb`, `cam_right_wrist_rgb` - **Camera Information:** cam_high_rgb; cam_high_realsense_rgb; cam_left_wrist_rgb; cam_right_wrist_rgb - **Scene:** `restaurant` - **Objects:** `table(unknown)`, `cheese(unknown)`, `lettuce(unknown)`, `bread(unknown)`, `plate(unknown)`, `sliced_bread(unknown)`, `meat_pie(unknown)`, `tomato_slices(unknown)` - **Task Description:** Make a hamburger with lettuce with the gripper., Make a hamburger with cheese with the gripper., Place the sliced bread, cheese, lettuce, meat patties, and tomato slices on a plate with the gripper., Make a hamburger with cheese and lettuce with the gripper. ### Primary Task Instruction > Make a hamburger with lettuce with the gripper., Make a hamburger with cheese with the gripper., Place the sliced bread, cheese, lettuce, meat patties, and tomato slices on a plate with the gripper., Make a hamburger with cheese and lettuce with the gripper. ### Robot Configuration - **Robot Name:** `Agilex_Cobot_Magic` - **Codebase Version:** `v2.1` - **End-Effector Type:** `two_finger_end_effector` - **Teleoperation Type:** `Due to some reasons, this dataset temporarily cannot provide the teleoperation type information.` ## Scene and Objects ### Scene Type - `restaurant` ### Objects - `table(unknown)` - `cheese(unknown)` - `lettuce(unknown)` - `bread(unknown)` - `plate(unknown)` - `sliced_bread(unknown)` - `meat_pie(unknown)` - `tomato_slices(unknown)` ## Task Descriptions - **Standardized Task Description:** `Make a hamburger with lettuce with the gripper., Make a hamburger with cheese with the gripper., Place the sliced bread, cheese, lettuce, meat patties, and tomato slices on a plate with the gripper., Make a hamburger with cheese and lettuce with the gripper.` - **Operation Type:** `Due to some reasons, this dataset temporarily cannot provide the operation type information.` - **Environment Type:** `Due to some reasons, this dataset temporarily cannot provide the environment type information.` ### Sub-Tasks This dataset includes 50 distinct subtasks: 1. **Place the purple cabbage on the cutlet with the left gripper.** (Index: 0) 2. **Pick up the tomato cut with the left gripper.** (Index: 1) 3. **Right pick up the patty.** (Index: 2) 4. **Right place the lettuce leaf on the bread cut.** (Index: 3) 5. **Right place the patty on the tomato cut.** (Index: 4) 6. **Place the hamburger lid on the cutlet with the right gripper.** (Index: 5) 7. **end.** (Index: 6) 8. **Pick up the bread slices and place them on the tray with the left gripper.** (Index: 7) 9. **Pick up the purple cabbage with the left gripper.** (Index: 8) 10. **Left pick up the bottom bread cut.** (Index: 9) 11. **Place the lettuce leaf on the bread cut with the right gripper.** (Index: 10) 12. **Pick up the bread cut with the left gripper.** (Index: 11) 13. **Left pick up the cheese cut.** (Index: 12) 14. **Discard.** (Index: 13) 15. **Pick up the cheese cut with the right gripper.** (Index: 14) 16. **Pick up the tomato cut with the right gripper.** (Index: 15) 17. **Place the hamburger lid on the cheese cut with the right gripper.** (Index: 16) 18. **End.** (Index: 17) 19. **Left place the bottom bread cut on the tray.** (Index: 18) 20. **Left place the tomato cut on the lettuce leaf.** (Index: 19) 21. **Abnormal.** (Index: 20) 22. **Place the purple cabbage on the tomato cut with the left gripper.** (Index: 21) 23. **Place the tomato cut on the lettuce leaf with the right gripper.** (Index: 22) 24. **Left pick up the tomato cut.** (Index: 23) 25. **Pick up the lettuce leaf with the right gripper.** (Index: 24) 26. **Place the tomato cut on the lettuce leaf with the left gripper.** (Index: 25) 27. **Place the hamburger lid on the purple cabbage with the right gripper.** (Index: 26) 28. **Place the tomato cut on the bread cut with the right gripper.** (Index: 27) 29. **Place the cutlet on the tomato cut with the right gripper.** (Index: 28) 30. **Place the hamburger lid on the cutlet cut with the left gripper.** (Index: 29) 31. **Place the cheese cut on the cutlet with the left gripper.** (Index: 30) 32. **Place the bread cut on the tray with the left gripper.** (Index: 31) 33. **Place the cutlet on the lettuce leaf with the right gripper.** (Index: 32) 34. **Left place the cheese cut on the patty.** (Index: 33) 35. **Place the hamburger lid on the cheese cut with the left gripper.** (Index: 34) 36. **Pick up the cutlet with the right gripper.** (Index: 35) 37. **Left place the bread on the tray.** (Index: 36) 38. **Pick up the hamburger lid with the right gripper.** (Index: 37) 39. **Place the cheese cut on the cutlet with the right gripper.** (Index: 38) 40. **Pick up the hamburger lid with the left gripper.** (Index: 39) 41. **Right pick up the top bread cut.** (Index: 40) 42. **Pick up the meat patty and place it on top of the tomato with the right gripper.** (Index: 41) 43. **abnormal.** (Index: 42) 44. **Pick up the cheese cut with the left gripper.** (Index: 43) 45. **Right pick up the lettuce leaf.** (Index: 44) 46. **Right place the lettuce leaf on the bottom bread.** (Index: 45) 47. **Left pick up the bottom bread.** (Index: 46) 48. **Right place the top bread cut on the cheese cut.** (Index: 47) 49. **Place the tomato cut on the bread cut with the left gripper.** (Index: 48) 50. **null.** (Index: 49) ### Atomic Actions - `grasp` - `place` - `pick` ## Hardware and Sensors ### Sensors - `cam_high_rgb` - `cam_high_realsense_rgb` - `cam_left_wrist_rgb` - `cam_right_wrist_rgb` ### Camera Information - `cam_high_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=av1, pix_fmt=yuv420p - `cam_high_realsense_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=av1, pix_fmt=yuv420p - `cam_left_wrist_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=av1, pix_fmt=yuv420p - `cam_right_wrist_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=av1, pix_fmt=yuv420p ### Coordinate System - **Definition:** `right-hand-frame` ### Dimensions & Units - **Joint Rotation:** `radian` - **End-Effector Rotation:** `radian` - **End-Effector Translation:** `meter` ## Dataset Statistics | Metric | Value | |--------|-------| | **Total Episodes** | 3044 | | **Total Frames** | 1591383 | | **Total Tasks** | 50 | | **Total Videos** | 12176 | | **Total Chunks** | 4 | | **Chunk Size** | 1000 | | **FPS** | 30 | | **State Dimensions** | 26 | | **Action Dimensions** | 26 | | **Camera Views** | 4 | | **Dataset Size** | 46.71 GB | ## Data Splits The dataset is organized into the following splits: - **Training**: Episodes 0:3043 ## Dataset Structure This dataset follows the LeRobot format and contains the following components: ### Data Files - **Videos**: Compressed video files containing RGB camera observations - **State Data**: Robot joint positions, velocities, and other state information - **Action Data**: Robot action commands and trajectories - **Metadata**: Episode metadata, timestamps, and annotations ### File Organization - **Data Path Pattern**: `data/chunk-{id}/episode_{id}.parquet` - **Video Path Pattern**: `videos/chunk-{id}/observation.images.cam_high_rgb/episode_{id}.mp{id}` - **Chunking**: Data is organized into 4 chunk(s) of size 1000 ### Data Structure (Tree) ``` Cobot_Magic_make_hamburger_qced_hardlink/ |-- annotations | |-- eef_acc_mag_annotation.jsonl | |-- eef_direction_annotation.jsonl | |-- eef_velocity_annotation.jsonl | |-- gripper_activity_annotation.jsonl | |-- gripper_mode_annotation.jsonl | |-- scene_annotations.jsonl | `-- subtask_annotations.jsonl |-- data | |-- chunk-000 | | |-- episode_000000.parquet | | |-- episode_000001.parquet | | |-- episode_000002.parquet | | |-- episode_000003.parquet | | |-- episode_000004.parquet | | |-- episode_000005.parquet | | |-- episode_000006.parquet | | |-- episode_000007.parquet | | |-- episode_000008.parquet | | |-- episode_000009.parquet | | |-- episode_000010.parquet | | `-- episode_000011.parquet | | `-- ... (988 more entries) | |-- chunk-001 | | |-- episode_001000.parquet | | |-- episode_001001.parquet | | |-- episode_001002.parquet | | |-- episode_001003.parquet | | |-- episode_001004.parquet | | |-- episode_001005.parquet | | |-- episode_001006.parquet | | |-- episode_001007.parquet | | |-- episode_001008.parquet | | |-- episode_001009.parquet | | |-- episode_001010.parquet | | `-- episode_001011.parquet | | `-- ... (988 more entries) | |-- chunk-002 | | |-- episode_002000.parquet | | |-- episode_002001.parquet | | |-- episode_002002.parquet | | |-- episode_002003.parquet | | |-- episode_002004.parquet | | |-- episode_002005.parquet | | |-- episode_002006.parquet | | |-- episode_002007.parquet | | |-- episode_002008.parquet | | |-- episode_002009.parquet | | |-- episode_002010.parquet | | `-- episode_002011.parquet | | `-- ... (988 more entries) | `-- chunk-003 | |-- episode_003000.parquet | |-- episode_003001.parquet | |-- episode_003002.parquet | |-- episode_003003.parquet | |-- episode_003004.parquet | |-- episode_003005.parquet | |-- episode_003006.parquet | |-- episode_003007.parquet | |-- episode_003008.parquet | |-- episode_003009.parquet | |-- episode_003010.parquet | `-- episode_003011.parquet | `-- ... (65 more entries) |-- meta | |-- episodes.jsonl | |-- episodes_stats.jsonl | |-- info.json | `-- tasks.jsonl |-- videos | |-- chunk-000 | | |-- observation.images.cam_high_realsense_rgb | | |-- observation.images.cam_high_rgb | | |-- observation.images.cam_left_wrist_rgb | | `-- observation.images.cam_right_wrist_rgb | |-- chunk-001 | | |-- observation.images.cam_high_realsense_rgb | | |-- observation.images.cam_high_rgb | | |-- observation.images.cam_left_wrist_rgb | | `-- observation.images.cam_right_wrist_rgb | |-- chunk-002 | | |-- observation.images.cam_high_realsense_rgb | | |-- observation.images.cam_high_rgb | | |-- observation.images.cam_left_wrist_rgb | | `-- observation.images.cam_right_wrist_rgb | `-- chunk-003 | |-- observation.images.cam_high_realsense_rgb | |-- observation.images.cam_high_rgb | |-- observation.images.cam_left_wrist_rgb | `-- observation.images.cam_right_wrist_rgb `-- README.md ``` ## Camera Views This dataset includes 4 camera views: `cam_high_rgb`, `cam_high_realsense_rgb`, `cam_left_wrist_rgb`, `cam_right_wrist_rgb`. ## Features (Full YAML) ```yaml observation.images.cam_high_rgb: dtype: video shape: - 480 - 640 - 3 names: - height - width - channels info: video.height: 480 video.width: 640 video.codec: av1 video.pix_fmt: yuv420p video.is_depth_map: false video.fps: 30 video.channels: 3 has_audio: false observation.images.cam_high_realsense_rgb: dtype: video shape: - 480 - 640 - 3 names: - height - width - channels info: video.height: 480 video.width: 640 video.codec: av1 video.pix_fmt: yuv420p video.is_depth_map: false video.fps: 30 video.channels: 3 has_audio: false observation.images.cam_left_wrist_rgb: dtype: video shape: - 480 - 640 - 3 names: - height - width - channels info: video.height: 480 video.width: 640 video.codec: av1 video.pix_fmt: yuv420p video.is_depth_map: false video.fps: 30 video.channels: 3 has_audio: false observation.images.cam_right_wrist_rgb: dtype: video shape: - 480 - 640 - 3 names: - height - width - channels info: video.height: 480 video.width: 640 video.codec: av1 video.pix_fmt: yuv420p video.is_depth_map: false video.fps: 30 video.channels: 3 has_audio: false observation.state: dtype: float32 shape: - 26 names: - left_arm_joint_1_rad - left_arm_joint_2_rad - left_arm_joint_3_rad - left_arm_joint_4_rad - left_arm_joint_5_rad - left_arm_joint_6_rad - left_gripper_open - left_eef_pos_x_m - left_eef_pos_y_m - left_eef_pos_z_m - left_eef_rot_euler_x_rad - left_eef_rot_euler_y_rad - left_eef_rot_euler_z_rad - right_arm_joint_1_rad - right_arm_joint_2_rad - right_arm_joint_3_rad - right_arm_joint_4_rad - right_arm_joint_5_rad - right_arm_joint_6_rad - right_gripper_open - right_eef_pos_x_m - right_eef_pos_y_m - right_eef_pos_z_m - right_eef_rot_euler_x_rad - right_eef_rot_euler_y_rad - right_eef_rot_euler_z_rad action: dtype: float32 shape: - 26 names: - left_arm_joint_1_rad - left_arm_joint_2_rad - left_arm_joint_3_rad - left_arm_joint_4_rad - left_arm_joint_5_rad - left_arm_joint_6_rad - left_gripper_open - left_eef_pos_x_m - left_eef_pos_y_m - left_eef_pos_z_m - left_eef_rot_euler_x_rad - left_eef_rot_euler_y_rad - left_eef_rot_euler_z_rad - right_arm_joint_1_rad - right_arm_joint_2_rad - right_arm_joint_3_rad - right_arm_joint_4_rad - right_arm_joint_5_rad - right_arm_joint_6_rad - right_gripper_open - right_eef_pos_x_m - right_eef_pos_y_m - right_eef_pos_z_m - right_eef_rot_euler_x_rad - right_eef_rot_euler_y_rad - right_eef_rot_euler_z_rad timestamp: dtype: float32 shape: - 1 names: null frame_index: dtype: int64 shape: - 1 names: null episode_index: dtype: int64 shape: - 1 names: null index: dtype: int64 shape: - 1 names: null task_index: dtype: int64 shape: - 1 names: null subtask_annotation: names: null dtype: int32 shape: - 5 scene_annotation: names: null dtype: int32 shape: - 1 eef_sim_pose_state: names: - left_eef_pos_x - left_eef_pos_y - left_eef_pos_z - left_eef_ori_x - left_eef_ori_y - left_eef_ori_z - right_eef_pos_x - right_eef_pos_y - right_eef_pos_z - right_eef_ori_x - right_eef_ori_y - right_eef_ori_z dtype: float32 shape: - 12 eef_sim_pose_action: names: - left_eef_pos_x - left_eef_pos_y - left_eef_pos_z - left_eef_ori_x - left_eef_ori_y - left_eef_ori_z - right_eef_pos_x - right_eef_pos_y - right_eef_pos_z - right_eef_ori_x - right_eef_ori_y - right_eef_ori_z dtype: float32 shape: - 12 eef_direction_state: names: - left_eef_direction - right_eef_direction dtype: int32 shape: - 2 eef_direction_action: names: - left_eef_direction - right_eef_direction dtype: int32 shape: - 2 eef_velocity_state: names: - left_eef_velocity - right_eef_velocity dtype: int32 shape: - 2 eef_velocity_action: names: - left_eef_velocity - right_eef_velocity dtype: int32 shape: - 2 eef_acc_mag_state: names: - left_eef_acc_mag - right_eef_acc_mag dtype: int32 shape: - 2 eef_acc_mag_action: names: - left_eef_acc_mag - right_eef_acc_mag dtype: int32 shape: - 2 gripper_open_scale_state: names: - left_gripper_open_scale - right_gripper_open_scale dtype: float32 shape: - 2 gripper_open_scale_action: names: - left_gripper_open_scale - right_gripper_open_scale dtype: float32 shape: - 2 gripper_mode_state: names: - left_gripper_mode - right_gripper_mode dtype: int32 shape: - 2 gripper_mode_action: names: - left_gripper_mode - right_gripper_mode dtype: int32 shape: - 2 gripper_activity_state: names: - left_gripper_activity - right_gripper_activity dtype: int32 shape: - 2 ``` ## Available Annotations This dataset includes rich annotations to support diverse learning approaches: - `eef_acc_mag_annotation.jsonl` - `eef_direction_annotation.jsonl` - `eef_velocity_annotation.jsonl` - `gripper_activity_annotation.jsonl` - `gripper_mode_annotation.jsonl` - `scene_annotations.jsonl` - `subtask_annotations.jsonl` ## Dataset Tags - `RoboCOIN` - `LeRobot` ## Authors ### Contributors This dataset is contributed by:-RoboCOIN Team at Beijing Academy of Artificial Intelligence (BAAI) ### Annotators No annotator information available. ## Links - **Homepage:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/) - **Paper:** [https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441) - **Repository:** [https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN) ## Contact and Support For questions, issues, or feedback regarding this dataset, please contact us. ### Support For technical support, please open an issue on our GitHub repository. ## License apache-2.0 ## Citation If you use this dataset in your research, please cite: ```bibtex @article{robocoin, title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation}, author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao}, journal={arXiv preprint arXiv:2511.17441}, url = {https://arxiv.org/abs/2511.17441}, year={2025}, } ``` ### Additional References If you use this dataset, please also consider citing: LeRobot Framework: https://github.com/huggingface/lerobot ## Version Information Initial Release