ankile/real01c-insert-marker-d1-baseline-uniform-r1-25k-nocf

--- license: apache-2.0 task_categories: - robotics tags: - LeRobot configs: - config_name: default data_files: data/*/*.parquet --- This dataset was created using [LeRobot](https://github.com/huggingface/lerobot). ## Dataset Description - **Homepage:** [More Information Needed] - **Paper:** [More Information Needed] - **License:** apache-2.0 ## Dataset Structure [meta/info.json](meta/info.json): ```json { "codebase_version": "v3.0", "robot_type": "franka", "total_episodes": 50, "total_frames": 13824, "total_tasks": 1, "chunks_size": 1000, "data_files_size_in_mb": 100, "video_files_size_in_mb": 200, "fps": 15, "splits": { "train": "0:50" }, "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet", "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4", "features": { "observation.state": { "dtype": "float32", "shape": [ 7 ], "names": [ "cart_pos_x", "cart_pos_y", "cart_pos_z", "cart_rot_x", "cart_rot_y", "cart_rot_z", "gripper_position" ] }, "action": { "dtype": "float32", "shape": [ 7 ], "names": [ "vel_x", "vel_y", "vel_z", "vel_roll", "vel_pitch", "vel_yaw", "gripper_action" ] }, "steps_to_go": { "dtype": "int64", "shape": [ 1 ], "names": [ "steps_to_go" ] }, "source": { "dtype": "int64", "shape": [ 1 ], "names": [ "source_id" ] }, "intervention": { "dtype": "int64", "shape": [ 1 ], "names": [ "intervention_flag" ] }, "success": { "dtype": "int64", "shape": [ 1 ], "names": [ "success_flag" ] }, "is_valid": { "dtype": "int64", "shape": [ 1 ], "names": [ "is_valid_flag" ] }, "reward": { "dtype": "float32", "shape": [ 1 ], "names": [ "reward" ] }, "done": { "dtype": "int64", "shape": [ 1 ], "names": [ "done_flag" ] }, "arm_id": { "dtype": "int64", "shape": [ 1 ], "names": [ "arm_id" ] }, "policy_id": { "dtype": "int64", "shape": [ 1 ], "names": [ "policy_id" ] }, "manifest_idx": { "dtype": "int64", "shape": [ 1 ], "names": [ "manifest_idx" ] }, "pen_x": { "dtype": "float32", "shape": [ 1 ], "names": [ "pen_x" ] }, "pen_y": { "dtype": "float32", "shape": [ 1 ], "names": [ "pen_y" ] }, "pen_yaw": { "dtype": "float32", "shape": [ 1 ], "names": [ "pen_yaw" ] }, "action.cartesian_velocity": { "dtype": "float32", "shape": [ 6 ], "names": [ "x", "y", "z", "roll", "pitch", "yaw" ] }, "action.cartesian_position": { "dtype": "float32", "shape": [ 6 ], "names": [ "x", "y", "z", "roll", "pitch", "yaw" ] }, "action.joint_velocity": { "dtype": "float32", "shape": [ 7 ], "names": [ "joint_0", "joint_1", "joint_2", "joint_3", "joint_4", "joint_5", "joint_6" ] }, "action.joint_position": { "dtype": "float32", "shape": [ 7 ], "names": [ "joint_0", "joint_1", "joint_2", "joint_3", "joint_4", "joint_5", "joint_6" ] }, "action.gripper_position": { "dtype": "float32", "shape": [ 1 ], "names": [ "gripper" ] }, "action.gripper_velocity": { "dtype": "float32", "shape": [ 1 ], "names": [ "gripper" ] }, "observation.state.cartesian_position": { "dtype": "float32", "shape": [ 6 ], "names": [ "x", "y", "z", "roll", "pitch", "yaw" ] }, "observation.state.joint_position": { "dtype": "float32", "shape": [ 7 ], "names": [ "joint_0", "joint_1", "joint_2", "joint_3", "joint_4", "joint_5", "joint_6" ] }, "observation.state.joint_velocity": { "dtype": "float32", "shape": [ 7 ], "names": [ "joint_0", "joint_1", "joint_2", "joint_3", "joint_4", "joint_5", "joint_6" ] }, "observation.state.cartesian_velocity": { "dtype": "float32", "shape": [ 6 ], "names": [ "x", "y", "z", "roll", "pitch", "yaw" ] }, "observation.state.gripper_position": { "dtype": "float32", "shape": [ 1 ], "names": [ "gripper" ] }, "observation.images.18650758_left": { "dtype": "video", "shape": [ 480, 640, 3 ], "names": [ "height", "width", "channels" ], "info": { "video.height": 480, "video.width": 640, "video.codec": "h264", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 15, "video.channels": 3, "has_audio": false } }, "observation.images.25916956_left": { "dtype": "video", "shape": [ 480, 640, 3 ], "names": [ "height", "width", "channels" ], "info": { "video.height": 480, "video.width": 640, "video.codec": "h264", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 15, "video.channels": 3, "has_audio": false } }, "timestamp": { "dtype": "float32", "shape": [ 1 ], "names": null }, "frame_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "episode_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "task_index": { "dtype": "int64", "shape": [ 1 ], "names": null } } } ``` ## Citation **BibTeX:** ```bibtex [More Information Needed] ```This dataset is a robotics control dataset created using LeRobot, collected with a Franka robot. It contains 50 training episodes, totaling 13,824 frames at 15fps. Features include robot state observations (e.g., Cartesian position, joint position, velocity), actions (e.g., Cartesian velocity, joint velocity), intervention flags, success flags, rewards, and video observations from two cameras (resolution 480x640). The dataset is designed for robot learning tasks, supporting reinforcement learning or imitation learning research.

搜集汇总

数据集介绍

构建方式

在机器人学习领域，高质量示范数据的采集与结构化是推动策略学习研究的基础。该数据集依托LeRobot框架构建，针对Franka机械臂平台，通过统一的数据采集流程记录了50个回合共13824帧的机械臂操作轨迹。数据以Parquet列式存储格式分块保存，每块容量上限100MB，视频文件则采用H.264编码的MP4格式独立存储，容量上限200MB。采集过程中同步记录了关节与笛卡尔空间下的位置、速度以及夹爪状态等多模态信息，并配套步骤剩余量、奖励、完成标志、干预标志等元数据，形成结构化的机器人示范数据集，为后续策略训练与评估提供统一的数据支撑。

特点

该数据集在结构设计上体现出多维状态表征与丰富元信息标注的双重特征。观测空间涵盖七自由度关节位置与速度、六维笛卡尔位姿与速度以及夹爪位置，动作空间同样包含关节与笛卡尔两个层面的位置与速度指令，并辅以笔尖坐标与偏航角等任务特定变量。数据集中同时提供steps_to_go、reward、done、success、intervention、is_valid等过程监督信号，以及source、policy_id、arm_id等来源标识，便于研究者开展离线策略评估、世界模型建模与模仿学习等任务。视频模态包含两路480×640分辨率、15帧率的左视角图像，为视觉-运动策略学习提供同步的多视角观测支持。

使用方法

在具体使用中，研究者可借助LeRobot工具链直接加载该数据集，依据meta/info.json中定义的数据路径与划分信息读取训练集全部50个回合。数据以chunk与file两级索引组织，可通过data_path与video_path模板定位对应的Parquet数据文件与MP4视频文件。针对模仿学习任务，可将observation.state与observation.images作为输入，action作为监督目标；针对离线强化学习或世界模型任务，则可联合使用reward、done与steps_to_go等信号。研究人员亦可依据intervention与success标志筛选高质量示范片段，或按policy_id与arm_id进行分组分析，从而灵活适配策略预训练、微调与评估等多类研究场景。

背景与挑战

背景概述

机器人学习领域长期面临高质量操作数据稀缺的瓶颈，推动社区构建标准化、可复现的数据资源。该数据集由LeRobot框架生成，于2024年前后发布，基于Franka机械臂平台，涵盖50个回合、13824帧、15Hz采样率，包含笛卡尔与关节空间的多模态状态、动作、奖励及成功标志。其核心问题在于为视觉-语言-动作模型提供细粒度操作轨迹，并支持插入标记物等精密任务研究。该数据集通过统一格式与开源许可，提升了机器人模仿学习与策略评估的可复现性，对具身智能社区的数据规范化具有示范意义。

当前挑战

该数据集所对应的领域问题——基于视觉的机器人精密插入操作，面临高精度末端位姿控制、接触力隐式建模以及视觉遮挡下状态估计等固有挑战。构建过程中，数据采集需协调Franka机械臂与双路摄像头同步，确保多模态观测的时间对齐；任务仅含单一操作类型，导致数据多样性受限，且缺少力/力矩传感器信息，对策略泛化能力提出考验；此外，50个回合的规模不足以覆盖复杂扰动场景，如何通过数据增强或仿真迁移弥补真实世界分布偏移，仍是后续研究需应对的关键难题。

常用场景

经典使用场景

在机器人学习与模仿学习领域，该数据集通常被用于训练和评估基于视觉-动作映射的端到端策略模型。其经典使用场景涵盖从多视角RGB观测中学习Franka机械臂的笛卡尔速度与夹爪控制指令，通过行为克隆或离线强化学习范式构建具备泛化能力的操作策略。数据集内嵌的steps_to_go、success、intervention等结构化元信息，使其特别适用于研究带有时序依赖和人类干预信号的策略学习问题。

实际应用

在实际应用层面，该数据集可服务于工业装配、实验室自动化及服务机器人等场景中的技能迁移与自适应控制。其7自由度动作空间与笛卡尔/关节空间双重视角的状态表征，使训练所得策略能够便捷地部署于Franka Emika Panda等真实硬件平台。同时，视频流与结构化状态信息的同步记录，为远程操作、示教编程以及数字孪生系统中的策略回放与仿真验证提供了高保真数据基础。

衍生相关工作

以该数据集为基础，后续研究多集中于干预感知的离线强化学习算法、基于视觉Transformer的机器人策略架构以及时序动作分块生成模型等方向。相关经典工作包括将steps_to_go作为辅助预测目标以提升长程规划能力的变体方法，以及利用policy_id与arm_id进行多策略融合与领域自适应的扩展研究，共同丰富了LeRobot生态下的可复现机器人学习基准。