MessyTable

我们提出了一个有趣且具有挑战性的数据集，其中包含大量场景以及从多个摄像机视图中捕获的凌乱表格。该数据集中的每个场景都非常复杂，包含多个对象实例，这些对象实例可能相同、堆叠并被其他实例遮挡。关键挑战是在给定所有视图的 RGB 图像的情况下关联所有实例。看似简单的任务令人惊讶地失败了许多我们假设在对象关联中表现良好的流行方法或启发式方法。该数据集在挖掘细微的外观差异、基于上下文的推理以及将外观与几何线索融合以建立关联方面挑战了现有方法。我们报告了一些流行基线的有趣发现，并讨论了该数据集如何帮助激发新问题并催化更强大的公式来解决现实世界的实例关联问题。

We present an intriguing and challenging dataset that includes numerous scenes featuring cluttered tabletops captured from multiple camera perspectives. Each scene in this dataset is highly complex, containing multiple object instances that may be identical, stacked, and occluded by other instances. The core challenge is to associate all object instances given the RGB images from all viewpoints. Many popular methods and heuristic approaches previously hypothesized to excel at object association surprisingly fail on this seemingly simple task. This dataset challenges existing methods by demanding them to discern subtle appearance variations, conduct context-aware reasoning, and fuse appearance and geometric cues to establish instance associations. We report intriguing findings from several popular baseline models, and discuss how this dataset can help inspire novel research questions and catalyze more robust formulations for addressing real-world instance association tasks.

搜集汇总

背景与挑战

背景概述

MessyTable是一个由商汤科技和南洋理工大学于2020年发布的数据集，包含多视角捕获的复杂凌乱表格场景，旨在通过RGB图像解决实例关联问题，挑战现有方法在外观差异、上下文推理和几何线索融合方面的能力。该数据集采用CC BY-NC-SA 3.0许可，可用于推动现实世界实例关联研究。