ObjectFolder 2.0

物体在我们的日常活动中起着至关重要的作用。尽管最近以多感官对象为中心的学习显示出巨大的潜力，但在以前的工作中进行对象建模是不现实的。ObjectFolder 1.0是一个最新的数据集，它通过视觉、听觉和触觉感觉数据引入100虚拟化对象。但是，数据集较小且多感官数据的质量有限，阻碍了对现实场景的概括。我们证明了ObjectFolder 2.0是隐式神经表示中表示的常见家庭对象的大规模，多感官数据集，可通过三种方式显着增强ObjectFolder 1.0。首先，我们的数据集的对象数量增加了10倍，渲染时间快了几个数量级。其次，我们在所有三种模式下都显着提高了多感官渲染的质量。第三，我们展示了从数据集中的虚拟对象学习的模型在三个具有挑战性的任务中成功地转移到现实世界中的对应对象: 对象规模估计，接触定位和形状重建。ObjectFolder 2.0为计算机视觉和机器人技术中的多感官学习提供了一种新的方法和测试平台。

Objects play a vital role in our daily activities. Although recent multi-sensory object-centric learning has shown great potential, object modeling in previous works was impractical. ObjectFolder 1.0 is a state-of-the-art dataset that introduces 100 virtualized objects with visual, auditory, and tactile sensory data. However, its small scale and limited quality of multi-sensory data hinder generalization to real-world scenarios. We demonstrate that ObjectFolder 2.0—a large-scale, multi-sensory dataset of common household objects represented via implicit neural representations—significantly enhances ObjectFolder 1.0 in three key aspects. First, the number of objects in our dataset is increased by 10-fold, and the rendering time is accelerated by orders of magnitude. Second, we significantly improve the quality of multi-sensory rendering across all three modalities. Third, we show that models trained on virtual objects from this dataset successfully transfer to their real-world counterparts across three challenging tasks: object scale estimation, contact localization, and shape reconstruction. ObjectFolder 2.0 provides a novel framework and testbed for multi-sensory learning research in computer vision and robotics.

搜集汇总

背景与挑战

背景概述

ObjectFolder 2.0是一个大规模多感官数据集，包含1000个虚拟化对象，提升了渲染质量和速度，支持视觉、听觉和触觉数据建模，并能在现实世界的对象规模估计、接触定位和形状重建任务中进行有效的转移学习。