MimicPlay
收藏资源简介:
MimicPlay 是由斯坦福大学、英伟达和佐治亚理工学院等机构联合开发的长时序机器人操作数据集,旨在通过人类玩耍数据提升机器人在复杂任务中的模仿学习能力。该数据集包含 6 个环境的 14 项长时序操作任务,涵盖厨房、书桌、花艺、白板、三明治制作和布料折叠等场景,任务时长为 2000 至 4000 步(约 100 - 200 秒)。数据集由两部分组成:一是 10 分钟的人类玩耍视频,记录人类自由与环境互动的 3D 手部轨迹;二是少量(20 - 40 条)机器人遥操作演示数据。数据集的创建过程结合了低成本的人类玩耍数据和少量高精度的机器人演示数据。人类玩耍数据通过多视角摄像头记录,利用现成的手部检测器重建 3D 手部轨迹。机器人演示数据则通过手机遥操作系统收集,涵盖多种任务目标。MimicPlay 的应用领域主要集中在机器人长时序任务的模仿学习,旨在解决复杂任务中数据需求量大、泛化能力差的问题。通过引入人类玩耍数据,该数据集能够显著提升机器人的样本效率和泛化能力,为机器人学习提供更丰富的语义信息和行为模式。
MimicPlay is a long-horizon robotic manipulation dataset jointly developed by institutions including Stanford University, NVIDIA, and Georgia Institute of Technology, aimed at enhancing the imitation learning capabilities of robots for complex tasks using human play data. This dataset includes 14 long-horizon manipulation tasks across 6 environments, covering scenarios such as kitchen, desk operation, floral arrangement, whiteboard interaction, sandwich preparation, and cloth folding, with each task lasting 2000 to 4000 steps (approximately 100 to 200 seconds). The dataset consists of two components: first, 10-minute human play videos that record 3D hand trajectories of humans freely interacting with the environment; second, a small number (20 to 40 instances) of robot teleoperation demonstration data. The development of this dataset combines low-cost human play data with a small amount of high-precision robot demonstration data. Human play data is recorded via multi-view cameras, and 3D hand trajectories are reconstructed using off-the-shelf hand detectors. Robot demonstration data, meanwhile, is collected via mobile phone teleoperation systems and covers a variety of task objectives. The primary application domain of MimicPlay is imitation learning for robotic long-horizon tasks, which aims to address the challenges of high data demand and poor generalization in complex robotic tasks. By introducing human play data, this dataset can significantly improve the sample efficiency and generalization ability of robots, providing richer semantic information and behavioral patterns for robot learning.




