POSITION: Open world 3D scene CAD recomposition

3D scene CAD recomposition aims to reconstruct agiven scene by retrieving and assembling CAD models from adatabase, so as to accurately simulate the geometric propertiesand spatial arrangement of the original environment. Recentmethods learn this task through training on limited scan-to-CADannotation data, which hinders their generalization to diversereal-world scenes. In this paper, we propose POSITION, anopen-world 3D scene CAD recomposition method to constructthe 3D scene with CADs retrieved from an open-set database.POSITION is designed following a divide-and-conquer strategy.Firstly, we extract open-world multi-modal object representationsfrom a captured 3D scene. Secondly, on top of the representations,we propose a coarse-to-fine retrieval method to retrieve CADsthat are visually, geometrically and semantically match realobjects. Thirdly, we present a physically plausible pose alignmentmethod to adjust retrieved CAD models to maintain consistentgeometry and layout with the observation. By decomposing theproblem into well-defined subtasks, our approach achieves gener-alization across various scene types and scalable CAD databaseswithout retraining or fine-tuning. Our approach demonstratessuperior CAD recomposition performance on both the Scan2CADbenchmark and diverse real-world 3D scene datasets.