Hypersim

对于许多基本的场景理解任务，很难或不可能从真实图像中获得每像素的地面真相标签。我们通过引入Hypersim来解决这一挑战，Hypersim是用于整体室内场景理解的真实感合成数据集。为了创建我们的数据集，我们利用专业艺术家创建的合成场景的大型存储库，我们生成461室内场景的77,400图像，并带有详细的每像素标签和相应的地面真相几何形状。我们的数据集 :( 1) 完全依赖于公开可用的3D资产; (2) 包括每个场景的完整场景几何、材料信息和照明信息; (3) 包括每个像素的密集语义实例分割和每个图像的完整相机信息；(4) 将每张图像分为漫反射，漫射照明和非漫射剩余项，该项可捕获与视图相关的照明效果。

For many fundamental scene understanding tasks, it is difficult or impossible to obtain per-pixel ground-truth labels from real-world images. We address this challenge by introducing Hypersim, a photorealistic synthetic dataset for holistic indoor scene understanding. To construct our dataset, we leverage a large repository of synthetic scenes created by professional artists, and generate 77,400 images across 461 indoor scenes, paired with detailed per-pixel labels and corresponding ground-truth geometry. Our dataset features four core properties: 1) It relies entirely on publicly available 3D assets; 2) It includes complete scene geometry, material information, and lighting information for every scene; 3) It provides dense semantic instance segmentation per pixel and full camera parameters for each image; 4) It decomposes each image into diffuse albedo, diffuse lighting, and a non-diffuse residual term that captures view-dependent lighting effects.

搜集汇总

背景与挑战

背景概述

Hypersim是一个用于整体室内场景理解的真实感合成数据集，包含461个室内场景的77,400张图像，提供每像素标签、地面真相几何、材料、照明信息以及密集语义实例分割。该数据集完全基于公开3D资产，旨在解决从真实图像中获取精确地面真相标签的挑战，适用于计算机视觉任务如语义分割和3D场景分析。