BrunoM42/robocasa_target_TurnOnElectricKettle

---
license: apache-2.0
task_categories:
- robotics
tags:
- LeRobot
configs:
- config_name: default
data_files: data/*/*.parquet
---

This dataset was created using [LeRobot](https://github.com/huggingface/lerobot).

## Dataset Description

- **Homepage:** [More Information Needed]
- **Paper:** [More Information Needed]
- **License:** apache-2.0

## Dataset Structure

[meta/info.json](meta/info.json):
```json
{
"codebase_version": "v3.0",
"robot_type": "PandaOmron",
"total_episodes": 520,
"total_frames": 84679,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 20,
"splits": {
"train": "0:520"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
"features": {
"observation.images.robot0_eye_in_hand": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.images.robot0_agentview_left": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.images.robot0_agentview_right": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"annotation.human.task_description": {
"dtype": "int64",
"shape": [
1
],
"fps": 20
},
"annotation.human.task_name": {
"dtype": "int64",
"shape": [
1
],
"fps": 20
},
"observation.state": {
"dtype": "float64",
"shape": [
16
],
"fps": 20
},
"action": {
"dtype": "float64",
"shape": [
12
],
"fps": 20
},
"next.reward": {
"dtype": "float32",
"shape": [
1
],
"fps": 20
},
"next.done": {
"dtype": "bool",
"shape": [
1
],
"fps": 20
},
"timestamp": {
"dtype": "float32",
"shape": [
1
],
"names": null,
"fps": 20
},
"frame_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"episode_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"task_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
}
},
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200
}
```

## Citation

**BibTeX:**

```bibtex
[More Information Needed]
```

license: apache-2.0
task_categories:
- robotics
tags:
- LeRobot(LeRobot)
configs:
- config_name: default
data_files: data/*/*.parquet
---

本数据集基于LeRobot(LeRobot)框架构建，项目仓库地址：https://github.com/huggingface/lerobot。

## 数据集描述

- **主页：** [需补充更多信息]
- **论文：** [需补充更多信息]
- **许可证：** apache-2.0

## 数据集结构

[meta/info.json](meta/info.json):
json
{
"codebase_version": "v3.0",
"robot_type": "PandaOmron",
"total_episodes": 520,
"total_frames": 84679,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 20,
"splits": {
"train": "0:520"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
"features": {
"observation.images.robot0_eye_in_hand": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.images.robot0_agentview_left": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.images.robot0_agentview_right": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 20,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
},
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "h264",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"annotation.human.task_description": {
"dtype": "int64",
"shape": [
1
],
"fps": 20
},
"annotation.human.task_name": {
"dtype": "int64",
"shape": [
1
],
"fps": 20
},
"observation.state": {
"dtype": "float64",
"shape": [
16
],
"fps": 20
},
"action": {
"dtype": "float64",
"shape": [
12
],
"fps": 20
},
"next.reward": {
"dtype": "float32",
"shape": [
1
],
"fps": 20
},
"next.done": {
"dtype": "bool",
"shape": [
1
],
"fps": 20
},
"timestamp": {
"dtype": "float32",
"shape": [
1
],
"names": null,
"fps": 20
},
"frame_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"episode_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
},
"task_index": {
"dtype": "int64",
"shape": [
1
],
"names": null,
"fps": 20
}
},
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200
}

## 引用

**BibTeX:**

bibtex
[需补充更多信息]

搜集汇总

数据集介绍

构建方式

在机器人操作学习领域，robocasa_target_TurnOnElectricKettle数据集依托LeRobot平台构建而成，其核心在于模拟真实环境中的电水壶开启任务。该数据集通过PandaOmron机器人采集了520个完整操作序列，总计84679帧数据，以每秒20帧的速率记录。数据以分块形式存储于Parquet文件中，每块包含1000帧，同时配套存储了对应的高清视频片段，确保了时序动作与视觉观测的同步对齐。

特点

该数据集的特点体现在多模态信息的深度融合，不仅提供了机器人手眼相机与全局视角的双目视觉流，还整合了16维的状态观测向量与12维的动作指令。所有视觉数据均以256x256分辨率的三通道图像呈现，采用h264编码保存，保证了数据的规范性与可处理性。数据集结构清晰，每个数据点均附有时戳、帧索引及任务标注，为端到端的模仿学习与强化学习算法提供了完备的输入输出对。

使用方法

使用robocasa_target_TurnOnElectricKettle数据集时，研究者可通过LeRobot框架直接加载Parquet格式的数据块，并依据帧索引关联相应的MP4视频文件。数据已预分割为训练集，涵盖全部520个序列，适用于行为克隆、视觉动作预测等任务的模型训练。在具体应用中，可同步调用观测图像、机器人状态与动作标签，以帧率为基准进行时序建模，进而优化机器人在家庭场景下的物体操作能力。

背景与挑战

背景概述

在机器人学习领域，模拟真实世界任务的数据集对于推动具身智能的发展至关重要。robocasa_target_TurnOnElectricKettle数据集由LeRobot团队创建，专注于机器人执行日常家务任务——开启电水壶。该数据集收录了520个完整交互序列，共计84679帧数据，采用PandaOmron机器人平台，以20帧每秒的速率捕捉多视角视觉观察与状态动作信息。其核心研究问题在于如何使机器人通过视觉与动作数据学习复杂的操作技能，从而提升在非结构化家庭环境中的自主执行能力。这一数据集为机器人模仿学习与强化学习算法提供了宝贵的训练资源，有助于缩小仿真与真实场景之间的鸿沟。

当前挑战

该数据集旨在解决机器人操作任务中的领域挑战，即让机器人在动态、非结构化的家庭环境中可靠地完成特定物品的操控。具体挑战包括：机器人需从多视角视觉输入中理解物体状态与空间关系，并生成精确的动作序列以应对物体形状、位置及物理特性的变化。在构建过程中，数据采集面临诸多困难，例如确保机器人动作的平滑性与安全性，同步记录高帧率多路视频与传感器数据，以及标注大规模时序数据时保持一致性。此外，真实世界任务的随机性与多样性要求数据集涵盖充分的任务变体，以增强模型的泛化能力，这对数据采集的规模与质量提出了较高要求。

常用场景

经典使用场景

在机器人操作学习领域，robocasa_target_TurnOnElectricKettle数据集为模拟家庭环境中的精细操作任务提供了关键资源。该数据集聚焦于让机器人执行打开电水壶这一日常动作，通过520个完整交互片段和超过8万帧的多视角视觉数据，捕捉了PandaOmron机械臂在真实场景下的动作序列与状态变化。研究者利用这些包含手眼相机和全局视角的高清视频流，结合动作、状态及奖励信号，能够训练模型理解并复现复杂的物体操控流程，为机器人学习人类日常任务中的灵巧操作奠定了数据基础。

解决学术问题

该数据集直接应对机器人模仿学习与强化学习中的核心挑战，即如何从高维视觉输入中有效提取操控策略。它通过提供结构化的多模态交互记录，解决了在非结构化家庭环境中动作表示学习、状态转移建模以及奖励函数设计等难题。其意义在于为学术界提供了一个可重复、标准化的基准，用以评估算法在真实世界物体交互任务上的泛化能力与鲁棒性，显著推动了从仿真到实际应用的过渡研究。

衍生相关工作

围绕该数据集衍生的经典工作主要集中在机器人操作策略的视觉表征学习与跨任务迁移上。例如，研究者利用其多视角视频序列开发了端到端的视觉运动策略网络，实现了从图像到动作的直接映射。同时，该数据集也常被纳入更大规模的机器人操作基准测试中，如与RoboSet或其他家庭任务数据集结合，用于评估元学习、多任务学习以及模仿学习框架在复杂长周期任务上的性能，催生了一系列关于样本效率与策略泛化的创新算法。