Rent3D

这项工作的目标是在给定一小组不同房间的单目图像以及 2D 平面图的情况下，实现公寓的 3D“虚拟游览”。我们将问题描述为马尔可夫随机场中的推断，该场推断每个房间的布局及其在整个公寓内的相对姿势（3D 旋转和平移）。这为我们提供了每个图像在公寓中的准确相机姿势。使我们在布局估计方面与过去工作的不同之处在于使用平面图作为先验知识的来源，以及在更大的空间（公寓）内定位每个图像。特别是，我们利用平面图对不同房间的布局施加纵横比约束，并提取语义信息，例如平面图中标记的窗户的位置。我们表明，这些信息可以显着帮助解决具有挑战性的房间-公寓对齐问题。我们还推导出了一个有效的精确推理算法，每个公寓只需要几毫秒。这是因为我们利用了积分几何以及我们对房间纵横比的新界限，这使我们能够雕刻空间，显着减少物理上可能的配置数量。我们在包含 200 多套公寓的新数据集上展示了我们的方法的有效性。

The goal of this work is to realize 3D "virtual tours" of apartments given a small set of monocular images of distinct rooms along with their corresponding 2D floor plans. We formulate the problem as inference in a Markov Random Field (MRF) that infers the layout of each room and their relative poses (3D rotation and translation) within the entire apartment, which yields accurate camera poses of each image within the apartment. What distinguishes our layout estimation approach from prior work is the use of floor plans as a source of prior knowledge, as well as the localization of each image within the larger spatial context of the entire apartment. In particular, we leverage floor plans to impose aspect ratio constraints on the layout of each distinct room, and extract semantic information such as the positions of windows marked in the floor plans. We demonstrate that this information can significantly help address the challenging room-to-apartment alignment problem. We also derive an efficient exact inference algorithm that takes only a few milliseconds per apartment. This is achieved by leveraging integral geometry and our novel bounds on room aspect ratios, which enable us to carve the search space and drastically reduce the number of physically feasible configurations. We validate the effectiveness of our method on a new dataset containing over 200 apartments.

搜集汇总

背景与挑战

背景概述

Rent3D是一个专注于从单目图像和2D平面图进行3D房间布局估计的数据集，包含200多套公寓数据，适用于计算机视觉中的语义分割和布局估计任务。该数据集由清华大学和多伦多大学联合发布，采用了先进的算法来提高推断的准确性和效率。