SeasonDepth: Cross-Season Monocular Depth Prediction Dataset

Changing environments poses great challenge on the outdoor visual perception and scene understanding for robust long-term autonomous driving and mobile robots, where depth-auxiliary geometric information plays an essential role to the robustness under challenging scenes. Although monocular depth prediction has been well studied recently, there are few work focused on the depth prediction across multiple environmental conditions, e.g. changing illumination and seasons, owing to the lack of such real-world dataset and benchmark. In this work, a new cross-season scaleless monocular depth prediction dataset SeasonDepth (Available on https://seasondepth.github.io) is derived from CMU Visual Localization dataset through structure from motion. To benchmark the depth estimation performance under different environments, we investigate representative and recent state-of-the-art open-source supervised, self-supervised and domain adaptation depth prediction methods from KITTI benchmark using several newly-formulated metrics. Through extensive experimental evaluation on the proposed dataset without fine-tuning, the influence of multiple environments on mean and variance of performance is analyzed, showing that the long-term monocular depth prediction is far from solved. We further give promising solutions especially with self-supervised stereo geometry and multi-task training to enhance the robustness to changing environments.

环境变化对户外视觉感知和场景理解构成了巨大挑战，尤其在实现稳健的长期自动驾驶和移动机器人方面。在此过程中，深度辅助几何信息在应对复杂场景中的稳健性方面扮演着至关重要的角色。尽管单目深度预测近期已得到广泛研究，但鉴于缺乏此类真实世界数据集和基准，针对多环境条件下（如光照变化和季节变化）的深度预测工作寥寥无几。在本研究中，我们通过结构光技术从 CMU 视觉定位数据集（可在 https://seasondepth.github.io 查阅）中衍生出一个新的跨季节无尺度单目深度预测数据集 SeasonDepth。为了在不同环境条件下对深度估计性能进行基准测试，我们采用数个新制定的指标，对 KITTI 基准测试中代表性的、最新的开源监督学习、自监督学习和领域自适应深度预测方法进行了研究。通过对所提出数据集的广泛实验评估，未进行微调的情况下，分析了多种环境对性能均值和方差的影响，表明长期单目深度预测问题尚未得到根本解决。此外，我们进一步提出了具有前景的解决方案，特别是采用自监督立体几何和多任务训练技术，以增强对环境变化的鲁棒性。