duck

# PiPER Duck Pick-and-Place Dataset **[简体中文](#简体中文) | [English](#english) | [Download & Loading](#download-and-loading)** AgileX PiPER 真机小黄鸭抓放示范数据集：双视角 RGB、7 维关节空间动作、LeRobot v2.1 格式。 Real-robot duck pick-and-place demonstrations with two RGB views and 7-dimensional joint-space commands, stored in LeRobot v2.1 format. ## 简体中文 ### 数据集简介 本数据集使用 AgileX PiPER 机械臂，通过主从示教采集桌面抓放轨迹。任务是抓起桌面上的小黄鸭，并将其放入圆柱形笔筒。机器人包含 6 个机械臂关节和 1 个夹爪自由度。 数据由 [Piper Studio](https://github.com/1024AILab/piper-studio) 采集，包含场景相机与腕部相机的 RGB 图像、主臂目标指令、上一帧指令状态以及任务标签，可用于模仿学习、视觉动作策略和 VLA 数据适配。 仓库内保存的任务文本为： > 抓起小黄鸭，放入笔筒 英文释义为 “Pick up the yellow duck and place it into the pen holder.”，该释义不替换数据中的中文标签。 ### 数据规模 以下统计对应当前发布的 `meta/info.json`、`meta/episodes.jsonl` 和数据文件清单。 | 项目 | 内容 | |---|---| | 数据集 ID | `czhczh666666/duck` | | 机器人 | AgileX PiPER，6 轴机械臂 + 1 自由度夹爪 | | 轨迹数量 | **202 episodes**，编号 `0–201` | | 总帧数 | **24,841** | | RGB 图像数 | **49,682**，每帧两个视角 | | 任务数量 | 1，`task_index = 0` | | 标称采样频率 | **10 FPS** | | 图像尺寸 | **640 × 480 RGB**；数组形状 `(480, 640, 3)` | | 单条轨迹长度 | 76–213 帧，按 10 FPS 折算约 7.6–21.3 秒 | | 总采样时长 | 约 **41.40 分钟**，按总帧数 / 10 FPS 折算 | | 数据格式 | LeRobot **v2.1** 元数据 + Parquet | | Parquet 文件体积 | 约 **22.57 GB**，共 202 个文件 | | 已声明划分 | `train: 0:202`；未提供独立验证集或测试集 | | 独立视频文件 | 0；图像以无损 PNG 字节嵌入 Parquet | ### 采集方式 主臂上的示教器驱动从臂执行任务，Piper Studio 监听共享 CAN 总线，读取主臂目标和从臂反馈。两路 Intel RealSense RGB 相机分别提供场景视角 `image` 和腕部视角 `wrist_image`，数据按 10 FPS 采样。 两台相机独立取帧，没有硬件同步触发。Parquet 的 `timestamp` 是从零开始的标称时间轴，即 `frame_index / 10`，不表示相机曝光时刻或真实采集延迟。采集平台将反馈与实际取样时刻写入数据集旁的 `.studio` 诊断目录；该诊断目录未包含在本次公开发布中。 ### 字段与动作语义 | 字段 | 类型 / 形状 | 含义 | |---|---|---| | `image` | RGB image，`(480, 640, 3)` | 场景视角；Parquet 中为含 PNG `bytes` 的结构体 | | `wrist_image` | RGB image，`(480, 640, 3)` | 腕部视角；存储方式同上 | | `state` | `float32[7]` | 上一帧记录的动作指令；首帧使用当前动作 | | `actions` | `float32[7]` | 当前绝对关节目标与夹爪目标 | | `timestamp` | `float32` | episode 内标称时间，单位为秒 | | `frame_index` | `int64` | episode 内从 0 开始的帧号 | | `episode_index` | `int64` | episode 编号 | | `index` | `int64` | 数据集内连续的全局帧号 | | `task_index` | `int64` | 关联 `meta/tasks.jsonl` 的任务编号 | `state` 与 `actions` 的元素顺序都是 `[joint_1, joint_2, joint_3, joint_4, joint_5, joint_6, gripper]`。前六维单位是 **0.001°**，第七维单位是 **µm**；文件中保存的是未归一化的原始目标值。 ```text state[0] = actions[0] state[t] = actions[t - 1] for t > 0 joint_degrees = raw_joint_value / 1000 joint_radians = raw_joint_value * pi / 180000 gripper_meters = raw_gripper_value / 1000000 ``` **这里的 `state` 表示上一帧指令，不是从臂的实测关节反馈。** `actions` 是绝对关节空间目标，不是末端位姿、增量动作或归一化到 `[0, 1]` 的夹爪开合比例。原始夹爪值含负值，训练时应保留明确的单位与预处理约定。 字段名保留为 `image / wrist_image / state / actions`。接入使用 `observation.images.*`、`observation.state` 或单数 `action` 的训练框架时，需要显式映射，并保持上述时序语义。数据集采样频率为 10 FPS；推理平台的 50 Hz 控制循环不改变这一数据频率。 许可证沿用仓库当前标注的 **Apache License 2.0**。 ## English ### Overview This dataset contains real-robot demonstrations of picking up a yellow duck from a tabletop and placing it into a cylindrical pen holder. Demonstrations were collected through leader–follower teleoperation on an AgileX PiPER with six arm joints and a one-degree-of-freedom gripper, using [Piper Studio](https://github.com/1024AILab/piper-studio). Each frame includes scene and wrist RGB observations, a seven-dimensional absolute command, the previous recorded command as `state`, and indexing metadata. The dataset supports imitation learning, visuomotor policy training, and VLA data adaptation. The stored task label is **`抓起小黄鸭，放入笔筒`** (`task_index = 0`), meaning “Pick up the yellow duck and place it into the pen holder.” The English translation is explanatory; the stored label remains Chinese. ### Release statistics | Item | Value | |---|---| | Dataset ID | `czhczh666666/duck` | | Episodes | **202**, indexed `0–201` | | Frames | **24,841** | | RGB images | **49,682**, two views per frame | | Tasks | 1 | | Nominal sampling rate | **10 FPS** | | Image resolution | **640 × 480 RGB**, decoded shape `(480, 640, 3)` | | Episode length | 76–213 frames, approximately 7.6–21.3 seconds at 10 FPS | | Total nominal duration | Approximately **41.40 minutes**, calculated as frames / FPS | | Format | LeRobot **v2.1** metadata with Parquet episode files | | Parquet size | Approximately **22.57 GB** across 202 files | | Declared split | `train: 0:202`; no separate validation or test split | | Standalone videos | 0; lossless PNG bytes are embedded in Parquet | ### Collection and timing The leader and its teaching pendant drive the follower through the hardware CAN link. Piper Studio observes leader targets and follower feedback while two Intel RealSense RGB cameras provide scene and wrist views. The cameras sample independently without hardware-triggered synchronization. Stored timestamps use the nominal time axis `frame_index / 10` within each episode. They are not exposure timestamps or measurements of collection latency. Actual feedback and timing diagnostics are kept by the collection software in an adjacent `.studio` directory, which is not part of this public release. ### Schema and action semantics | Field | Type / shape | Meaning | |---|---|---| | `image` | RGB image, `(480, 640, 3)` | Scene view, stored as a struct containing PNG `bytes` | | `wrist_image` | RGB image, `(480, 640, 3)` | Wrist view, stored in the same way | | `state` | `float32[7]` | Previous recorded command; the first frame uses the current command | | `actions` | `float32[7]` | Current absolute joint and gripper targets | | `timestamp` | `float32` | Nominal time in seconds within the episode | | `frame_index` | `int64` | Zero-based frame index within the episode | | `episode_index` | `int64` | Episode identifier | | `index` | `int64` | Continuous global frame index | | `task_index` | `int64` | Task identifier linked to `meta/tasks.jsonl` | Both vectors follow `[joint_1, joint_2, joint_3, joint_4, joint_5, joint_6, gripper]`. Joint values use **0.001 degrees**; the gripper value uses **micrometers**. These are raw, unnormalized target values. ```text state[0] = actions[0] state[t] = actions[t - 1] for t > 0 joint_degrees = raw_joint_value / 1000 joint_radians = raw_joint_value * pi / 180000 gripper_meters = raw_gripper_value / 1000000 ``` **`state` is the previous command, not measured follower proprioception.** Actions are absolute joint-space targets, not Cartesian poses, deltas, or a gripper ratio in `[0, 1]`. Raw gripper values include negative values; preserve a documented unit and preprocessing convention. The original keys are `image / wrist_image / state / actions`. Frameworks expecting `observation.images.*`, `observation.state`, or `action` require an explicit adapter that preserves the temporal convention. The companion runtime's 50 Hz control loop does not change the dataset's 10 FPS sampling rate. The dataset retains the repository's existing **Apache License 2.0** designation. ## Download and Loading ### Repository layout / 文件结构 ```text duck/ ├── README.md ├── data/ │ └── chunk-000/ │ ├── episode_000000.parquet │ ├── ... │ └── episode_000201.parquet └── meta/ ├── info.json ├── tasks.jsonl ├── episodes.jsonl └── episodes_stats.jsonl ``` Each Parquet file stores one episode. `episodes.jsonl` records episode lengths and tasks; `episodes_stats.jsonl` contains per-episode statistics. Image records contain `bytes` and `path`, with PNG data in `bytes` and `path = null` in this release. The `video_path` template in `info.json` is unused because `total_videos = 0`. 每个 Parquet 文件对应一条 episode。图像已经嵌入文件，不需要另外寻找 MP4；`info.json` 中保留的 `video_path` 只是未使用的路径模板。 ### ModelScope SDK ```bash python -m pip install -U modelscope ``` The following downloads only metadata and the first episode, approximately 181 MB. Remove `allow_patterns` to download all episodes, approximately 22.57 GB. 以下示例只下载元数据和第一条轨迹，约 181 MB；删除 `allow_patterns` 参数即可下载全部数据，约 22.57 GB。 ```python from modelscope.hub.snapshot_download import dataset_snapshot_download dataset_root = dataset_snapshot_download( dataset_id="czhczh666666/duck", revision="master", local_dir="./duck", allow_patterns=[ "README.md", "meta/*", "data/chunk-000/episode_000000.parquet", ], ) print(dataset_root) ``` This partial download is sufficient for the raw Parquet example below. Download all episodes before using a loader that expects every file declared in `meta/info.json`. 部分下载可用于下面的 Parquet 读取示例。若加载器要求元数据声明的所有文件均存在，请先下载完整数据集。 For reproducible experiments, pin a commit revision. The data statistics above were checked against revision `15a2bbb5e2dbf62ea685e18858c308d3999cf6a4`. ### Git LFS With Git LFS installed / 安装 Git LFS 后： ```bash git lfs install git clone https://modelscope.cn/datasets/czhczh666666/duck.git ``` The clone downloads the full dataset through LFS. Small text files beginning with `version https://git-lfs.github.com/spec/v1` are LFS pointers, not the dataset contents; run `git lfs pull` inside the cloned repository if the large files were skipped. Git 克隆默认通过 LFS 下载全量数据。如果得到的是以 `version https://git-lfs.github.com/spec/v1` 开头的小文本指针文件，请在仓库目录运行 `git lfs pull` 获取实际文件。 ### Read one episode / 读取一条轨迹 ```bash python -m pip install pyarrow pillow numpy ``` ```python from io import BytesIO from pathlib import Path import numpy as np import pyarrow.parquet as pq from PIL import Image root = Path("./duck") path = root / "data/chunk-000/episode_000000.parquet" parquet = pq.ParquetFile(path) # Numeric columns are small; no image decoding is needed for this read. signals = parquet.read(columns=["state", "actions", "timestamp"]).to_pydict() state = np.asarray(signals["state"], dtype=np.float32) actions = np.asarray(signals["actions"], dtype=np.float32) np.testing.assert_array_equal(state[0], actions[0]) np.testing.assert_array_equal(state[1:], actions[:-1]) # Read one row group to inspect the first pair of PNG images. first = parquet.read_row_group( 0, columns=["image", "wrist_image"] ).to_pylist()[0] scene = Image.open(BytesIO(first["image"]["bytes"])).convert("RGB") wrist = Image.open(BytesIO(first["wrist_image"]["bytes"])).convert("RGB") joint_radians = actions[:, :6] * (np.pi / 180000.0) gripper_meters = actions[:, 6] / 1_000_000.0 print("Frames:", parquet.metadata.num_rows) print("Image sizes:", scene.size, wrist.size) # (640, 480), (640, 480) print("Action shape:", actions.shape) # (episode_length, 7) ``` ## Dataset Citation ```bibtex @misc{piper_duck_2026, author = {{czhczh666666}}, title = {{PiPER} Duck Pick-and-Place Dataset}, year = {2026}, howpublished = {ModelScope}, url = {https://modelscope.cn/datasets/czhczh666666/duck} } ```