RoboCOIN/Agilex_Cobot_Magic_place_towel_flat

---
task_categories:
- robotics

language:
- en

extra_gated_prompt: 'By accessing this dataset, you agree to cite the associated paper in your research/publications—see the "Citation" section for details. You agree to not use the dataset to conduct experiments that cause harm to human subjects.'

extra_gated_fields:

Company/Organization:
type: 'text'
description: 'e.g., "ETH Zurich", "Boston Dynamics", "Independent Researcher"'

Country:
type: 'country'
description: 'e.g., "Germany", "China", "United States"'

tags:
- RoboCOIN
- LeRobot

license: apache-2.0

configs:
- config_name: default
data_files: data/chunk-{id}/episode_{id}.parquet
---

# Agilex_Cobot_Magic_place_towel_flat

## Dataset Description

This dataset uses an extended format based on LeRobot and is fully compatible with LeRobot.

## Task Preview

<video src="videos/chunk-000/observation.images.cam_head_rgb/episode_000000.mp4" controls width="640"></video>

[View Video Directly](videos/chunk-000/observation.images.cam_head_rgb/episode_000000.mp4)

### Overview

- **Total Episodes:** 962
- **Total Frames:** 1023212
- **FPS:** 30
- **Dataset Size:** 62.89 GB
- **Robot Name:** `Agilex_Cobot_Magic`
- **End-Effector Type:** `two_finger_gripper`
- **Teleoperation Type:** `Due to some reasons, this dataset temporarily cannot provide the teleoperation type information.`
- **Sensors:** `cam_head_rgb`,
`cam_right_wrist_rgb`,
`cam_left_wrist_rgb`

- **Camera Information:** cam_head_rgb;
cam_right_wrist_rgb;
cam_left_wrist_rgb

- **Scene:** `office_workspace->office`
- **Objects:** `table(unknown)`,
`basket(unknown)`,
`blue_towel(unknown)`

- **Task Description:** take out the towel from the basket and lay it flat on the table.

### Primary Task Instruction
> take out the towel from the basket and lay it flat on the table.

### Robot Configuration

- **Robot Name:** `Agilex_Cobot_Magic`
- **Codebase Version:** `v2.1`
- **End-Effector Type:** `two_finger_gripper`
- **Teleoperation Type:** `Due to some reasons, this dataset temporarily cannot provide the teleoperation type information.`

## Scene and Objects

### Scene Type
`office_workspace->office`

### Objects
- `table(unknown)`
- `basket(unknown)`
- `blue_towel(unknown)`

## Task Descriptions

- **Standardized Task Description:** `take out the towel from the basket and lay it flat on the table.`
- **Operation Type:** `Due to some reasons, this dataset temporarily cannot provide the operation type information.`

- **Environment Type:** `Due to some reasons, this dataset temporarily cannot provide the environment type information.`

### Sub-Tasks
This dataset includes 29 distinct subtasks:

1. **Right hand:adjust the blue towel** (Index: 0)
2. **Left hand: grab the brown towel** (Index: 1)
3. **Left hand: lift the blue towel to the center of view** (Index: 2)
4. **Right hand: grab a corner of the blue towel and straighten it** (Index: 3)
5. **Right hand: spread the blue towel flat on the table** (Index: 4)
6. **Right hand: grab a corner of the purple towel and straighten it** (Index: 5)
7. **Left hand: spread the brown towel flat on the table** (Index: 6)
8. **Right hand: spread the purple towel flat on the table** (Index: 7)
9. **Right hand: spread the grey towel flat on the table** (Index: 8)
10. **Left hand: spread the purple towel flat on the table** (Index: 9)
11. **Left hand:adjust the gray towel** (Index: 10)
12. **Left hand:adjust the brown towel** (Index: 11)
13. **Abnormal** (Index: 12)
14. **Right hand: grab a corner of the grey towel and straighten it** (Index: 13)
15. **Left hand: grab the grey towel** (Index: 14)
16. **Left hand: lift the purple towel to the center of view** (Index: 15)
17. **Right hand:adjust the brown towel** (Index: 16)
18. **Left hand: lift the brown towel to the center of view** (Index: 17)
19. **Left hand: grab the blue towel** (Index: 18)
20. **Left hand: lift the grey towel to the center of view** (Index: 19)
21. **Right hand: spread the brown towel flat on the table** (Index: 20)
22. **Left hand:adjust the blue towel** (Index: 21)
23. **Right hand: grab a corner of the brown towel and straighten it** (Index: 22)
24. **Left hand: spread the grey towel flat on the table** (Index: 23)
25. **End** (Index: 24)
26. **Right hand:adjust the gray towel** (Index: 25)
27. **Left hand: spread the blue towel flat on the table** (Index: 26)
28. **Left hand: grab the purple towel** (Index: 27)
29. **null** (Index: 28)

### Atomic Actions
- `grasp`
- `fold`
- `lift`
- `lower`

## Hardware and Sensors

### Sensors

- `cam_head_rgb`

- `cam_right_wrist_rgb`

- `cam_left_wrist_rgb`

### Camera Information

- `cam_head_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=h264, pix_fmt=yuv420p

- `cam_right_wrist_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=h264, pix_fmt=yuv420p

- `cam_left_wrist_rgb`: dtype=video, shape=480x640x3, resolution=640x480, codec=h264, pix_fmt=yuv420p

### Coordinate System
- **Definition:** `right-hand-frame`

### Dimensions & Units
- **Joint Rotation:** `radian`
- **End-Effector Rotation:** `radian`
- **End-Effector Translation:** `meter`

## Dataset Statistics

| Metric | Value |
|--------|-------|
| **Total Episodes** | 962 |
| **Total Frames** | 1023212 |
| **Total Tasks** | 29 |
| **Total Videos** | 2886 |
| **Total Chunks** | 1 |
| **Chunk Size** | 10000 |
| **FPS** | 30 |
| **State Dimensions** | 26 |
| **Action Dimensions** | 26 |
| **Camera Views** | 3 |
| **Dataset Size** | 62.89 GB |

## Data Splits

The dataset is organized into the following splits:

- **Training**: Episodes 0:961
- **Validation**: Episodes 795:894
- **Test**: Episodes 894:994

## Dataset Structure

This dataset follows the LeRobot format and contains the following components:

### Data Files
- **Videos**: Compressed video files containing RGB camera observations
- **State Data**: Robot joint positions, velocities, and other state information
- **Action Data**: Robot action commands and trajectories
- **Metadata**: Episode metadata, timestamps, and annotations

### File Organization
- **Data Path Pattern**: `data/chunk-{id}/episode_{id}.parquet`
- **Video Path Pattern**: `videos/chunk-{id}/observation.images.cam_head_rgb/episode_{id}.mp{id}`
- **Chunking**: Data is organized into 1 chunk(s)
of size 10000

### Data Structure (Tree)

```
Agilex_Cobot_Magic_place_towel_flat_qced_hardlink/
|-- annotations
| |-- eef_acc_mag_annotation.jsonl
| |-- eef_direction_annotation.jsonl
| |-- eef_velocity_annotation.jsonl
| |-- gripper_activity_annotation.jsonl
| |-- gripper_mode_annotation.jsonl
| |-- scene_annotations.jsonl
| `-- subtask_annotations.jsonl
|-- data
| `-- chunk-000
| |-- episode_000000.parquet
| |-- episode_000001.parquet
| |-- episode_000002.parquet
| |-- episode_000003.parquet
| |-- episode_000004.parquet
| |-- episode_000005.parquet
| |-- episode_000006.parquet
| |-- episode_000007.parquet
| |-- episode_000008.parquet
| |-- episode_000009.parquet
| |-- episode_000010.parquet
| `-- episode_000011.parquet
| `-- ... (950 more entries)
|-- meta
| |-- episodes.jsonl
| |-- episodes_stats.jsonl
| |-- info.json
| `-- tasks.jsonl
`-- videos
`-- chunk-000
|-- observation.images.cam_head_rgb
|-- observation.images.cam_left_wrist_rgb
`-- observation.images.cam_right_wrist_rgb
```

## Camera Views

This dataset includes 3 camera views: `cam_head_rgb`, `cam_right_wrist_rgb`, `cam_left_wrist_rgb`.

## Features (Full YAML)

```yaml
action:
dtype: float32
shape:
- 26
names:
- left_arm_joint_1_rad
- left_arm_joint_2_rad
- left_arm_joint_3_rad
- left_arm_joint_4_rad
- left_arm_joint_5_rad
- left_arm_joint_6_rad
- left_eef_pos_x_m
- left_eef_pos_y_m
- left_eef_pos_z_m
- left_eef_rot_euler_x_rad
- left_eef_rot_euler_y_rad
- left_eef_rot_euler_z_rad
- left_gripper_open
- right_arm_joint_1_rad
- right_arm_joint_2_rad
- right_arm_joint_3_rad
- right_arm_joint_4_rad
- right_arm_joint_5_rad
- right_arm_joint_6_rad
- right_eef_pos_x_m
- right_eef_pos_y_m
- right_eef_pos_z_m
- right_eef_rot_euler_x_rad
- right_eef_rot_euler_y_rad
- right_eef_rot_euler_z_rad
- right_gripper_open
observation.state:
dtype: float32
shape:
- 26
names:
- left_arm_joint_1_rad
- left_arm_joint_2_rad
- left_arm_joint_3_rad
- left_arm_joint_4_rad
- left_arm_joint_5_rad
- left_arm_joint_6_rad
- left_eef_pos_x_m
- left_eef_pos_y_m
- left_eef_pos_z_m
- left_eef_rot_euler_x_rad
- left_eef_rot_euler_y_rad
- left_eef_rot_euler_z_rad
- left_gripper_open
- right_arm_joint_1_rad
- right_arm_joint_2_rad
- right_arm_joint_3_rad
- right_arm_joint_4_rad
- right_arm_joint_5_rad
- right_arm_joint_6_rad
- right_eef_pos_x_m
- right_eef_pos_y_m
- right_eef_pos_z_m
- right_eef_rot_euler_x_rad
- right_eef_rot_euler_y_rad
- right_eef_rot_euler_z_rad
- right_gripper_open
observation.images.cam_head_rgb:
dtype: video
shape:
- 480
- 640
- 3
names:
- height
- width
- channels
info:
video.fps: 30.0
video.height: 480
video.width: 640
video.channels: 3
video.codec: h264
video.pix_fmt: yuv420p
video.is_depth_map: false
has_audio: false
observation.images.cam_right_wrist_rgb:
dtype: video
shape:
- 480
- 640
- 3
names:
- height
- width
- channels
info:
video.fps: 30.0
video.height: 480
video.width: 640
video.channels: 3
video.codec: h264
video.pix_fmt: yuv420p
video.is_depth_map: false
has_audio: false
observation.images.cam_left_wrist_rgb:
dtype: video
shape:
- 480
- 640
- 3
names:
- height
- width
- channels
info:
video.fps: 30.0
video.height: 480
video.width: 640
video.channels: 3
video.codec: h264
video.pix_fmt: yuv420p
video.is_depth_map: false
has_audio: false
timestamp:
dtype: float32
shape:
- 1
names: null
frame_index:
dtype: int64
shape:
- 1
names: null
episode_index:
dtype: int64
shape:
- 1
names: null
index:
dtype: int64
shape:
- 1
names: null
task_index:
dtype: int64
shape:
- 1
names: null
subtask_annotation:
names: null
dtype: int32
shape:
- 5
scene_annotation:
names: null
dtype: int32
shape:
- 1
eef_sim_pose_state:
names:
- left_eef_pos_x
- left_eef_pos_y
- left_eef_pos_z
- left_eef_rot_x
- left_eef_rot_y
- left_eef_rot_z
- right_eef_pos_x
- right_eef_pos_y
- right_eef_pos_z
- right_eef_rot_x
- right_eef_rot_y
- right_eef_rot_z
dtype: float32
shape:
- 12
eef_sim_pose_action:
names:
- left_eef_pos_x
- left_eef_pos_y
- left_eef_pos_z
- left_eef_rot_x
- left_eef_rot_y
- left_eef_rot_z
- right_eef_pos_x
- right_eef_pos_y
- right_eef_pos_z
- right_eef_rot_x
- right_eef_rot_y
- right_eef_rot_z
dtype: float32
shape:
- 12
eef_direction_state:
names:
- left_eef_direction
- right_eef_direction
dtype: int32
shape:
- 2
eef_direction_action:
names:
- left_eef_direction
- right_eef_direction
dtype: int32
shape:
- 2
eef_velocity_state:
names:
- left_eef_velocity
- right_eef_velocity
dtype: int32
shape:
- 2
eef_velocity_action:
names:
- left_eef_velocity
- right_eef_velocity
dtype: int32
shape:
- 2
eef_acc_mag_state:
names:
- left_eef_acc_mag
- right_eef_acc_mag
dtype: int32
shape:
- 2
eef_acc_mag_action:
names:
- left_eef_acc_mag
- right_eef_acc_mag
dtype: int32
shape:
- 2
gripper_mode_state:
names:
- left_gripper_mode
- right_gripper_mode
dtype: int32
shape:
- 2
gripper_mode_action:
names:
- left_gripper_mode
- right_gripper_mode
dtype: int32
shape:
- 2
gripper_activity_state:
names:
- left_gripper_activity
- right_gripper_activity
dtype: int32
shape:
- 2
gripper_activity_action:
names:
- left_gripper_activity
- right_gripper_activity
dtype: int32
shape:
- 2
gripper_open_scale_state:
names:
- left_gripper_open_scale
- right_gripper_open_scale
dtype: float32
shape:
- 2
gripper_open_scale_action:
names:
- left_gripper_open_scale
- right_gripper_open_scale
dtype: float32
shape:
- 2

```

## Available Annotations

This dataset includes rich annotations to support diverse learning approaches:

- `eef_acc_mag_annotation.jsonl`
- `eef_direction_annotation.jsonl`
- `eef_velocity_annotation.jsonl`
- `gripper_activity_annotation.jsonl`
- `gripper_mode_annotation.jsonl`
- `scene_annotations.jsonl`
- `subtask_annotations.jsonl`

## Dataset Tags

- `RoboCOIN`
- `LeRobot`

## Authors

### Contributors
This dataset is contributed by:-RoboCOIN Team at Beijing Academy of Artificial Intelligence (BAAI)

### Annotators
No annotator information available.

## Links

- **Homepage:** [https://flagopen.github.io/RoboCOIN/](https://flagopen.github.io/RoboCOIN/)
- **Paper:** [https://arxiv.org/abs/2511.17441](https://arxiv.org/abs/2511.17441)
- **Repository:** [https://github.com/FlagOpen/RoboCOIN](https://github.com/FlagOpen/RoboCOIN)
## Contact and Support

For questions, issues, or feedback regarding this dataset, please contact us.
### Support
For technical support, please open an issue on our GitHub repository.

## License

apache-2.0

## Citation

If you use this dataset in your research, please cite:

```bibtex
@article{robocoin,
title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
journal={arXiv preprint arXiv:2511.17441},
url = {https://arxiv.org/abs/2511.17441},
year={2025},
}

```

### Additional References

If you use this dataset, please also consider citing:
LeRobot Framework: https://github.com/huggingface/lerobot

## Version Information

Initial Release

搜集汇总

数据集介绍

构建方式

在机器人操作学习领域，高质量数据集的构建对于推动算法发展至关重要。Agilex_Cobot_Magic_place_towel_flat数据集基于LeRobot框架进行了扩展，确保了格式的完全兼容性。该数据集通过记录Agilex_Cobot_Magic双臂机器人在办公场景中执行“从篮子中取出毛巾并将其平铺在桌上”任务的过程而构建。数据采集依托于三路高帧率RGB摄像头，分别部署于机器人头部及双腕，以每秒30帧的速率捕捉了总计962条任务轨迹，生成了超过百万帧的视觉与状态序列。数据以分块形式组织，存储于Parquet格式文件中，并辅以详尽的结构化标注，为模仿学习与强化学习研究提供了扎实的多模态基础。

特点

该数据集在机器人操作数据集中展现出鲜明的技术特色。其核心在于对复杂双臂协同操作的细致记录，涵盖了29种明确的子任务与抓取、折叠、提升、放下等原子动作。数据维度丰富，不仅包含26维的关节状态与动作空间，还提供了末端执行器的位姿、速度、加速度及夹爪活动模式等多层次标注。三路视角同步的视觉流以640x480分辨率呈现，确保了操作场景的全方位覆盖。数据集规模达62.89GB，包含训练、验证与测试的标准划分，其严谨的右手法则坐标系定义与统一的物理单位（弧度与米制）进一步保障了数据的规范性与可直接用于算法训练的实用性。

使用方法

为有效利用该数据集进行机器人学习研究，使用者可遵循其既定的数据结构。数据按分块组织，可通过解析指定路径下的Parquet文件加载每一幕（episode）的序列数据，其中整合了观测状态、动作指令、时间戳及丰富的注释信息。配套的视频文件独立存储，需根据文件树结构进行关联读取。研究既可基于原始的高维状态与动作数据训练策略模型，也可利用其细粒度的子任务与场景标注进行分层或条件化学习。数据集完全兼容LeRobot生态系统，便于直接接入现有的数据处理与训练流程。在使用前，用户需同意相关的引用协议，并注意数据中部分元信息字段的暂时缺失。

背景与挑战

背景概述

在机器人操作学习领域，构建高质量、大规模的真实世界数据集是推动具身智能发展的关键基石。Agilex_Cobot_Magic_place_towel_flat数据集由北京智源人工智能研究院（BAAI）的RoboCOIN团队于2025年贡献，作为RoboCOIN项目的一部分，旨在解决双臂协作机器人执行复杂、长周期操作任务的学习难题。该数据集聚焦于一项具体的日常操作任务——从篮子中取出毛巾并将其平整铺放在桌面上，通过记录Agilex Cobot Magic机器人在办公室环境下的962次完整操作片段，提供了超过一百万帧的多视角视觉观察、高维状态与动作数据。其核心研究问题在于如何通过真实交互数据，赋能机器人学习对非刚性物体进行灵巧、顺序化的双手操作策略，从而推动模仿学习与强化学习算法在现实场景中的泛化与应用。

当前挑战

该数据集致力于解决机器人操作中非刚性物体灵巧操控这一经典挑战。具体而言，任务要求机器人不仅需精准抓取柔软、易变形的毛巾，还需协调双臂完成展开、铺平等一系列精细动作，这对动作的时序规划、双手协同以及基于视觉的实时状态估计提出了极高要求。在数据集构建层面，挑战同样显著：首先，采集真实机器人双臂操作数据成本高昂，需确保硬件系统的长期稳定与同步，并处理多路高清视频流带来的巨大数据量（约62.89GB）。其次，对长周期、多步骤任务进行精细标注（如29个子任务与原子动作的划分）需要大量人工介入，且需保证标注的一致性与准确性。此外，数据中存在的操作异常片段与部分信息缺失（如遥操作类型未提供），也为后续的数据清洗与算法训练带来了额外的复杂性。

常用场景

经典使用场景

在机器人操作学习领域，Agilex_Cobot_Magic_place_towel_flat数据集为双手机器人执行精细布料操作任务提供了标准化的演示数据。该数据集的核心任务是从篮子中取出毛巾并将其平整铺在桌面上，这一过程涉及抓取、提升、调整与铺展等一系列复杂的灵巧操作。研究者通常利用该数据集训练端到端的模仿学习或强化学习模型，使机器人能够从多视角视觉观察中理解任务意图，并生成协调的双臂动作序列，以完成对可变形物体的精确操控。

解决学术问题

该数据集主要致力于解决机器人操作中针对可变形物体进行精细、长序列任务规划的学术挑战。它通过提供大量真实世界的双手机器人演示数据，为研究多模态感知融合、分层策略学习以及动作序列的时序建模提供了关键资源。其意义在于弥合了仿真环境与真实物理世界之间的鸿沟，推动了从单一刚性物体抓取到复杂可变形物体操作的范式转变，对提升机器人在家庭服务、工业装配等场景中的自主性和适应性具有深远影响。

衍生相关工作

基于此数据集及其所属的RoboCOIN项目，已衍生出多项围绕大规模机器人操作数据收集、表示学习与策略泛化的经典研究工作。这些工作探索了如何利用此类多视角视频与状态动作对数据，进行视觉运动策略的预训练、跨任务的知识迁移以及少样本模仿学习。相关研究进一步推动了LeRobot等开源机器人学习框架的发展，为构建通用化的机器人操作基础模型提供了重要的数据支柱和算法验证基准。