Austin VIOLA

VIOLA，这是一种以对象为中心的模仿学习方法，用于学习机器人操作的闭环视觉运动策略。我们的方法基于预先训练的视觉模型中的一般对象建议构建以对象为中心的表示。它使用基于转换器的策略来推理这些表示，并关注与任务相关的视觉因素以进行操作预测。这种基于对象的结构先验提高了深度模仿学习算法对对象变化和环境扰动的鲁棒性。我们在模拟和真实机器人上定量评估VIOLA。VIOLA的成功率比最先进的模仿学习方法高出45.8%。它还已成功部署在物理机器人上，以解决具有挑战性的长距离任务，例如餐桌布置和咖啡制作。

VIOLA is an object-centric imitation learning approach for learning closed-loop visual motor policies for robotic manipulation. Our method constructs object-centric representations based on general object proposals from pre-trained vision models. It utilizes transformer-based policies to reason over these representations and focuses on task-relevant visual factors for manipulation prediction. This object-based structural prior enhances the robustness of deep imitation learning algorithms against object variations and environmental perturbations. We quantitatively evaluate VIOLA both in simulation and on real robotic platforms. VIOLA achieves a 45.8% higher success rate compared to state-of-the-art imitation learning methods. It has also been successfully deployed on physical robots to solve challenging long-horizon tasks such as table setting and coffee preparation.

搜集汇总

背景与挑战

背景概述

Austin VIOLA是一个用于机器人操作的以对象为中心的模仿学习数据集，基于预训练视觉模型构建对象表示，并采用Transformer策略进行推理。该方法在模拟和真实环境中表现出色，比现有方法提高了45.8%的成功率，已成功应用于餐桌布置和咖啡制作等任务。