DobbE

Dobb·E 是由纽约大学和 Meta 等机构联合开发的面向家庭机器人的操作数据集，旨在使机器人能够快速适应家庭环境中的多样化任务。该数据集通过一种名为“The Stick”的低成本演示工具收集，利用 iPhone 和 3D 打印组件，以极低的成本在纽约市的 22 个家庭中收集了 13 小时的演示数据，涵盖 5620 条轨迹，包含超过 150 万帧图像。数据集中的每个样本均包含 RGB 图像、深度信息以及 6D 位姿数据，为机器人提供了丰富的视觉和空间信息。数据集的创建过程充分考虑了家庭环境的复杂性和多样性。研究团队在不同家庭中进行数据收集，涵盖了多种常见的家庭任务，如开关门、抽屉操作、物体拾取与放置等。通过这种方式，Dobb·E 数据集能够为机器人提供广泛的任务类型和场景，使其能够更好地适应真实家庭环境中的各种挑战。Dobb·E is a manipulation dataset for home robots jointly developed by institutions including New York University and Meta, aiming to empower robots to rapidly adapt to diverse tasks within home environments. This dataset is collected via a low-cost demonstration tool named "The Stick", which utilizes iPhone devices and 3D-printed components. At an exceptionally low cost, the research team collected 13 hours of demonstration data across 22 households in New York City, encompassing 5620 trajectories and over 1.5 million image frames. Each sample in the dataset includes RGB images, depth information, and 6D pose data, providing rich visual and spatial information for robotic systems. The development of the Dobb·E dataset fully considers the complexity and diversity of real-world home environments. The research team conducted data collection across various households, covering a wide range of common domestic tasks such as opening and closing doors, drawer manipulation, object picking and placing, and more. Through this approach, the Dobb·E dataset can provide robots with a broad spectrum of task types and scenarios, enabling them to better adapt to various challenges in real home environments.

搜集汇总

背景与挑战

背景概述

Dobb·E是一个开源的家庭机器人操作学习框架，包含来自22个纽约家庭的13小时交互数据，涵盖109个任务，成功率达81%。数据集提供RGB和深度视频及动作注释，支持家庭机器人模仿学习研究。