CMU Food Manipulation

机器人食品操作的一个关键挑战是对各种可变形食品的材料特性进行建模。我们建议使用多模式感官方法与食物互动和玩耍，以促进区分食物中这些特性的能力。首先，我们使用机械臂和一系列传感器（使用 ROS 进行同步）来收集多样化的数据集，其中包含 21 种具有不同切片和特性的独特食品。之后，我们学习视觉嵌入网络，该网络利用本体感受、音频和视觉数据的组合，使用三元组损失公式对食物之间的相似性进行编码。我们的评估表明，通过交互学习的嵌入可以成功提高各种材料和形状分类任务的性能。我们设想这些学习的嵌入可以用作规划和选择最佳参数的基础，以实现更多材料感知的机器人食品操作技能。此外，我们希望通过与研究界分享这个食品游戏数据集来刺激食品机器人领域的进一步创新。

A core challenge in robotic food manipulation is modeling the material properties of various deformable food items. We propose a multimodal sensory approach for interacting and exploring with foods to enhance the ability to differentiate these material properties across different food products. First, we employ a robotic arm and a set of synchronized sensors (using ROS) to collect a diverse dataset encompassing 21 unique food items with distinct cuts and inherent properties. Subsequently, we train a visual embedding network that leverages the combined proprioceptive, audio, and visual data to encode the similarity between food items via a triplet loss formulation. Our evaluations demonstrate that embeddings learned through interactive exploration can effectively improve the performance of diverse material and shape classification tasks. We envision that these learned embeddings can serve as a basis for planning and selecting optimal parameters to enable more material-aware robotic food manipulation skills. Furthermore, we aim to stimulate further innovation in the food robotics domain by sharing this Food Play Dataset with the global research community.

搜集汇总

背景与挑战

背景概述

该数据集专注于机器人食品操作，通过多模态感官数据（视觉、音频、本体感受）学习食品特性，包含21种不同食品的数据，旨在提升材料感知的机器人操作技能。