Cobot_Magic_classification_of_fruits_and_vegetables

# Cobot_Magic_classification_of_fruits_and_vegetables

## 📋 Overview

This dataset uses an extended format based on LeRobot and is fully compatible with LeRobot.

**Robot Type:** `Cobot_Magic`
| **Codebase Version:** `v2.1`
**End-Effector Type:** `two_finger_gripper`

## 🏠 Scene Types

This dataset covers the following scene types:
- `home`

## 🤖 Atomic Actions

This dataset includes the following atomic actions:
- `grasp`
- `pick`
- `place`

## 📊 Dataset Statistics

| Metric | Value |
|--------|-------|
| **Total Episodes** | 301 |
| **Total Frames** | 183132 |
| **Total Tasks** | 4 |
| **Total Videos** | 903 |
| **Total Chunks** | 1 |
| **Chunk Size** | 1000 |
| **FPS** | 30 |

## 👥 Authors

### Contributors
This dataset is contributed by:
- [RoboCOIN](https://flagopen.github.io/RoboCOIN/) - RoboCOIN Team

## 🔗 Links

- **🏠 Homepage:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **📄 Paper:** [https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441)
- **💻 Repository:** [https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN)
- **🌐 Project Page:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **🐛 Issues:** [https://github.com/FlagOpen/RoboCOIN/issues](https://github.com/FlagOpen/RoboCOIN/issues)
- **📜 License:** apache-2.0

## 🏷️ Dataset Tags

- `RoboCOIN`
- `LeRobot`

## 🎯 Task Descriptions

### Primary Tasks
on blue tablecloth alternately place white carrots eggplants in plates of different colors.
on brown tablecloth alternately place white carrots eggplants in plates of different colors.
on green tablecloth alternately place white carrots eggplants in plates of different colors.
on pink tablecloth alternately place white carrots eggplants in plates of different colors.

### Sub-Tasks
This dataset includes 18 distinct subtasks:

1. **null**
2. **use the left arm to grab a carrot and put it into the left plate**
3. **use the left arm to grab a carrot and put it into the right plate**
4. **use the left arm to grab a chayote and put it into the left plate**
5. **use the left arm to grab a chayote and put it into the right plate**
6. **use the left arm to grab a radish and put it into the left plate**
7. **use the left arm to grab a radish and put it into the right plate**
8. **use the left arm to grab a tomato and put it into the left plate**
9. **use the left arm to grab an eggplant and put it into the left plate**
10. **use the left arm to grab an eggplant and put it into the right plate**
11. **use the right arm to grab a carrot and put it into the left plate**
12. **use the right arm to grab a carrot and put it into the right plate**
13. **use the right arm to grab a chayote and put it into the right plate**
14. **use the right arm to grab a radish and put it into the left plate**
15. **use the right arm to grab a radish and put it into the right plate**
16. **use the right arm to grab a tomato and put it into the left plate**
17. **use the right arm to grab an eggplant and put it into the left plate**
18. **use the right arm to grab an eggplant and put it into the right plate**

## 🎥 Camera Views

This dataset includes 3 camera views.

## 🏷️ Available Annotations

This dataset includes rich annotations to support diverse learning approaches:

### Subtask Annotations
- **Subtask Segmentation**: Fine-grained subtask segmentation and labeling
### Scene Annotations
- **Scene-level Descriptions**: Semantic scene classifications and descriptions
### End-Effector Annotations
- **Direction**: Movement direction classifications for robot end-effectors
- **Velocity**: Velocity magnitude categorizations during manipulation
- **Acceleration**: Acceleration magnitude classifications for motion analysis

### Gripper Annotations
- **Gripper Mode**: Open/close state annotations for gripper control
- **Gripper Activity**: Activity state classifications (active/inactive)

### Additional Features
- **End-Effector Simulation Pose**: 6D pose information for end-effectors in simulation space
- Available for both state and action
- **Gripper Opening Scale**: Continuous gripper opening measurements
- Available for both state and action

## 📂 Data Splits

The dataset is organized into the following splits:

- **Training**: Episodes 0:300

## 📁 Dataset Structure

This dataset follows the LeRobot format and contains the following components:

### Data Files
- **Videos**: Compressed video files containing RGB camera observations
- **State Data**: Robot joint positions, velocities, and other state information
- **Action Data**: Robot action commands and trajectories
- **Metadata**: Episode metadata, timestamps, and annotations

### File Organization
- **Data Path Pattern**: `data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet`
- **Video Path Pattern**: `videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4`
- **Chunking**: Data is organized into 1 chunk(s)
of size 1000

### Features Schema

The dataset includes the following features:

#### Visual Observations
- **observation.images.cam_front_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_left_wrist_rgb**: video
- FPS: 30
- Codec: av1- **observation.images.cam_right_wrist_rgb**: video
- FPS: 30
- Codec: av1

#### State and Action- **observation.state**: float32- **action**: float32

#### Temporal Information
- **timestamp**: float32
- **frame_index**: int64
- **episode_index**: int64
- **index**: int64
- **task_index**: int64

#### Annotations
- **subtask_annotation**: int32
- **scene_annotation**: int32

#### Motion Features
- **eef_sim_pose_state**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_sim_pose_action**: float32
- Dimensions: left_eef_pos_x, left_eef_pos_y, left_eef_pos_z, left_eef_ori_x, left_eef_ori_y, left_eef_ori_z, right_eef_pos_x, right_eef_pos_y, right_eef_pos_z, right_eef_ori_x, right_eef_ori_y, right_eef_ori_z
- **eef_direction_state**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_direction_action**: int32
- Dimensions: left_eef_direction, right_eef_direction
- **eef_velocity_state**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_velocity_action**: int32
- Dimensions: left_eef_velocity, right_eef_velocity
- **eef_acc_mag_state**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag
- **eef_acc_mag_action**: int32
- Dimensions: left_eef_acc_mag, right_eef_acc_mag

#### Gripper Features
- **gripper_open_scale_state**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_open_scale_action**: float32
- Dimensions: left_gripper_open_scale, right_gripper_open_scale
- **gripper_mode_state**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_mode_action**: int32
- Dimensions: left_gripper_mode, right_gripper_mode
- **gripper_activity_state**: int32
- Dimensions: left_gripper_activity, right_gripper_activity

### Meta Information

The complete dataset metadata is available in [meta/info.json](meta/info.json):

```json
{"codebase_version": "v2.1", "robot_type": "agilex_cobot_decoupled_magic", "total_episodes": 301, "total_frames": 183132, "total_tasks": 4, "total_videos": 903, "total_chunks": 1, "chunks_size": 1000, "fps": 30, "splits": {"train": "0:300"}, "data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet", "video_path": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4", "features": {"observation.images.cam_front_rgb": {"dtype": "video", "shape": [720, 1280, 3], "names": ["height", "width", "channels"], "info": {"video.height": 720, "video.width": 1280, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_left_wrist_rgb": {"dtype": "video", "shape": [720, 1280, 3], "names": ["height", "width", "channels"], "info": {"video.height": 720, "video.width": 1280, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.images.cam_right_wrist_rgb": {"dtype": "video", "shape": [720, 1280, 3], "names": ["height", "width", "channels"], "info": {"video.height": 720, "video.width": 1280, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false}}, "observation.state": {"dtype": "float32", "shape": [14], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open"]}, "action": {"dtype": "float32", "shape": [14], "names": ["left_arm_joint_1_rad", "left_arm_joint_2_rad", "left_arm_joint_3_rad", "left_arm_joint_4_rad", "left_arm_joint_5_rad", "left_arm_joint_6_rad", "left_gripper_open", "right_arm_joint_1_rad", "right_arm_joint_2_rad", "right_arm_joint_3_rad", "right_arm_joint_4_rad", "right_arm_joint_5_rad", "right_arm_joint_6_rad", "right_gripper_open"]}, "timestamp": {"dtype": "float32", "shape": [1], "names": null}, "frame_index": {"dtype": "int64", "shape": [1], "names": null}, "episode_index": {"dtype": "int64", "shape": [1], "names": null}, "index": {"dtype": "int64", "shape": [1], "names": null}, "task_index": {"dtype": "int64", "shape": [1], "names": null}, "subtask_annotation": {"names": null, "dtype": "int32", "shape": [5]}, "scene_annotation": {"names": null, "dtype": "int32", "shape": [1]}, "eef_sim_pose_state": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_sim_pose_action": {"names": ["left_eef_pos_x", "left_eef_pos_y", "left_eef_pos_z", "left_eef_ori_x", "left_eef_ori_y", "left_eef_ori_z", "right_eef_pos_x", "right_eef_pos_y", "right_eef_pos_z", "right_eef_ori_x", "right_eef_ori_y", "right_eef_ori_z"], "dtype": "float32", "shape": [12]}, "eef_direction_state": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_direction_action": {"names": ["left_eef_direction", "right_eef_direction"], "dtype": "int32", "shape": [2]}, "eef_velocity_state": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_velocity_action": {"names": ["left_eef_velocity", "right_eef_velocity"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_state": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "eef_acc_mag_action": {"names": ["left_eef_acc_mag", "right_eef_acc_mag"], "dtype": "int32", "shape": [2]}, "gripper_open_scale_state": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_open_scale_action": {"names": ["left_gripper_open_scale", "right_gripper_open_scale"], "dtype": "float32", "shape": [2]}, "gripper_mode_state": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_mode_action": {"names": ["left_gripper_mode", "right_gripper_mode"], "dtype": "int32", "shape": [2]}, "gripper_activity_state": {"names": ["left_gripper_activity", "right_gripper_activity"], "dtype": "int32", "shape": [2]}}}
```

### Directory Structure

The dataset is organized as follows (showing leaf directories with first 5 files only):

```
Cobot_Magic_classification_of_fruits_and_vegetables_qced_hardlink/
├── annotations/
│ ├── eef_acc_mag_annotation.jsonl
│ ├── eef_direction_annotation.jsonl
│ ├── eef_velocity_annotation.jsonl
│ ├── gripper_activity_annotation.jsonl
│ ├── gripper_mode_annotation.jsonl
│ └── (...)
├── data/
│ └── chunk-000/
│ ├── episode_000000.parquet
│ ├── episode_000001.parquet
│ ├── episode_000002.parquet
│ ├── episode_000003.parquet
│ ├── episode_000004.parquet
│ └── (...)
├── meta/
│ ├── episodes.jsonl
│ ├── episodes_stats.jsonl
│ ├── info.json
│ └── tasks.jsonl
└── videos/
└── chunk-000/
├── observation.images.cam_front_rgb/
│ ├── episode_000000.mp4
│ ├── episode_000001.mp4
│ ├── episode_000002.mp4
│ ├── episode_000003.mp4
│ ├── episode_000004.mp4
│ └── (...)
├── observation.images.cam_left_wrist_rgb/
│ ├── episode_000000.mp4
│ ├── episode_000001.mp4
│ ├── episode_000002.mp4
│ ├── episode_000003.mp4
│ ├── episode_000004.mp4
│ └── (...)
└── observation.images.cam_right_wrist_rgb/
├── episode_000000.mp4
├── episode_000001.mp4
├── episode_000002.mp4
├── episode_000003.mp4
├── episode_000004.mp4
└── (...)
```

## 📞 Contact and Support

For questions, issues, or feedback regarding this dataset, please contact:
- **Email:** None
For questions, issues, or feedback regarding this dataset, please contact us.
### Support
For technical support, please open an issue on our GitHub repository.

## 📄 License

This dataset is released under the **apache-2.0** license.

Please refer to the LICENSE file for full license terms and conditions.

## 📚 Citation

If you use this dataset in your research, please cite:

```bibtex
@article{robocoin,
title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
journal={arXiv preprint arXiv:2511.17441},
url = {https://arxiv.org/abs/2511.17441},
year={2025}
}
```

### Additional References

If you use this dataset, please also consider citing:
- LeRobot Framework: https://github.com/huggingface/lerobot

## 📌 Version Information

## Version History
- v1.0.0 (2025-11): Initial release

# Cobot_Magic果蔬分类数据集

## 📋 概览
本数据集基于LeRobot扩展格式开发，且完全兼容LeRobot。
**机器人类型**：`Cobot_Magic`
| **代码库版本**：`v2.1`
**末端执行器类型**：`双指夹爪（two_finger_gripper）`

## 🏠 场景类型
本数据集涵盖以下场景类型：
- `home`

## 🤖 原子动作
本数据集包含以下原子动作：
- `grasp（抓取）`
- `pick（拾取）`
- `place（放置）`

## 📊 数据集统计
| 指标 | 数值 |
|--------|-------|
| **总回合数（Total Episodes）** | 301 |
| **总帧数（Total Frames）** | 183132 |
| **总任务数（Total Tasks）** | 4 |
| **总视频数（Total Videos）** | 903 |
| **总数据块数（Total Chunks）** | 1 |
| **数据块大小（Chunk Size）** | 1000 |
| **帧率（FPS）** | 30 |

## 👥 作者
### 贡献者
本数据集由以下团队贡献：
- [RoboCOIN](https://flagopen.github.io/RoboCOIN/) - RoboCOIN团队

## 🔗 相关链接
- **🏠 主页**：[https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **📄 论文**：[https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441)
- **💻 代码仓库**：[https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN)
- **🌐 项目页面**：[https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **🐛 问题反馈**：[https://github.com/FlagOpen/RoboCOIN/issues](https://github.com/FlagOpen/RoboCOIN/issues)
- **📜 开源协议**：apache-2.0

## 🎯 任务描述
### 主任务
1. 在蓝色桌布上交替将白胡萝卜、茄子放置到不同颜色的餐盘内。
2. 在棕色桌布上交替将白胡萝卜、茄子放置到不同颜色的餐盘内。
3. 在绿色桌布上交替将白胡萝卜、茄子放置到不同颜色的餐盘内。
4. 在粉色桌布上交替将白胡萝卜、茄子放置到不同颜色的餐盘内。

### 子任务
本数据集包含18个独立子任务：
1. **null（空任务）**
2. **使用左机械臂抓取一根胡萝卜，并将其放入左侧餐盘**
3. **使用左机械臂抓取一根胡萝卜，并将其放入右侧餐盘**
4. **使用左机械臂抓取一个佛手瓜，并将其放入左侧餐盘**
5. **使用左机械臂抓取一个佛手瓜，并将其放入右侧餐盘**
6. **使用左机械臂抓取一个萝卜，并将其放入左侧餐盘**
7. **使用左机械臂抓取一个萝卜，并将其放入右侧餐盘**
8. **使用左机械臂抓取一个番茄，并将其放入左侧餐盘**
9. **使用左机械臂抓取一个茄子，并将其放入左侧餐盘**
10. **使用左机械臂抓取一个茄子，并将其放入右侧餐盘**
11. **使用右机械臂抓取一根胡萝卜，并将其放入左侧餐盘**
12. **使用右机械臂抓取一根胡萝卜，并将其放入右侧餐盘**
13. **使用右机械臂抓取一个佛手瓜，并将其放入右侧餐盘**
14. **使用右机械臂抓取一个萝卜，并将其放入左侧餐盘**
15. **使用右机械臂抓取一个萝卜，并将其放入右侧餐盘**
16. **使用右机械臂抓取一个番茄，并将其放入左侧餐盘**
17. **使用右机械臂抓取一个茄子，并将其放入左侧餐盘**
18. **使用右机械臂抓取一个茄子，并将其放入右侧餐盘**

## 🎥 相机视角
本数据集包含3种相机视角。

## 🏷️ 可用标注
本数据集包含丰富的标注信息，可支撑多样化的学习研究方法：
### 子任务标注
- **子任务分割**：细粒度的子任务分割与标注
### 场景标注
- **场景级描述**：语义化的场景分类与描述
### 末端执行器标注
- **运动方向**：机器人末端执行器的运动方向分类
- **运动速度**：操作过程中的速度幅值分类
- **运动加速度**：用于运动分析的加速度幅值分类
### 夹爪标注
- **夹爪模式**：夹爪开合状态的标注
- **夹爪活动状态**：夹爪的活动/非活动状态分类
### 附加特征
- **末端执行器仿真位姿**：仿真空间中末端执行器的6D位姿信息，支持状态与动作两类数据
- **夹爪开度**：连续的夹爪开度测量值，支持状态与动作两类数据

## 📂 数据划分
本数据集按以下方式划分训练集：
- **训练集**：第0~300号回合

## 📄 开源协议
本数据集采用**apache-2.0**协议发布。完整的协议条款请参见LICENSE文件。

## 📞 联系与支持
如需咨询数据集相关问题、反馈bug或提出建议，请联系：
- **邮箱**：暂无
如需技术支持，请在GitHub仓库中提交issue。

## 📚 引用说明
若您在研究中使用本数据集，请引用以下文献：
bibtex
@article{robocoin,
title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
journal={arXiv preprint arXiv:2511.17441},
url = {https://arxiv.org/abs/2511.17441},
year={2025}
}

### 补充引用
若您使用本数据集，还请考虑引用以下文献：
- LeRobot框架：[https://github.com/huggingface/lerobot](https://github.com/huggingface/lerobot)

## 📌 版本信息
### 版本历史
- v1.0.0 (2025-11)：首次发布