The NeRF-Stereo Dataset

The dataset contains several image sequences collected with mobile phones and the corresponding image triplets and disparity labels for training deep stereo networks effortlessly and without any ground-truth. By leveraging state-of-the-art neural rendering solutions, we generate stereo training data from image sequences collected with a single handheld camera. On top of them, a NeRF-supervised training procedure is carried out, from which we exploit rendered stereo triplets to compensate for occlusions and depth maps as proxy labels. This results in stereo networks capable of predicting sharp and detailed disparity maps.

本数据集包含多组由移动设备采集的图像序列，以及配套的图像三元组与视差标签（disparity labels），可便捷用于深度立体网络（deep stereo networks）的训练，且无需额外提供任何真实标注（ground-truth）。借助当前最先进的神经渲染（neural rendering）技术方案，我们可通过单台手持相机采集的图像序列生成立体训练数据；在此基础上执行基于神经辐射场（NeRF）的监督训练流程，利用渲染得到的立体图像三元组对遮挡区域进行补偿，并将深度图作为代理标签（proxy labels），最终得到的立体网络可生成清晰且细节丰富的视差图。