SeasonDepth: Cross-Season Monocular Depth Prediction Training Dataset

Changing environments poses great challenge on the outdoor visual perception and scene understanding for robust long-term autonomous driving and mobile robots, where depth-auxiliary geometric information plays an essential role to the robustness under challenging scenes. Although monocular depth prediction has been well studied recently, there are few work focused on the depth prediction across multiple environmental conditions, e.g. changing illumination and seasons, owing to the lack of such real-world dataset and benchmark. In this work, a new cross-season scaleless monocular depth prediction dataset SeasonDepth (Available on https://seasondepth.github.io) is derived from CMU Visual Localization dataset through structure from motion. To benchmark the depth estimation performance under different environments, we investigate representative and recent state-of-the-art open-source supervised, self-supervised and domain adaptation depth prediction methods from KITTI benchmark using several newly-formulated metrics. Through extensive experimental evaluation on the proposed dataset without fine-tuning, the influence of multiple environments on mean and variance of performance is analyzed, showing that the long-term monocular depth prediction is far from solved. We further give promising solutions especially with self-supervised stereo geometry and multi-task training to enhance the robustness to changing environments.

环境变迁对户外视觉感知和场景理解构成了巨大的挑战，这对于实现稳健的长期自动驾驶和移动机器人至关重要。在复杂场景下，深度辅助几何信息在提高稳健性方面发挥着至关重要的作用。尽管单目深度预测最近已得到广泛研究，但由于缺乏此类真实世界数据集和基准，针对跨多种环境条件（例如变化的光照和季节）的深度预测研究寥寥无几。在本研究中，我们通过结构光从 CMU 视觉定位数据集中派生出一个新的跨季节无尺度单目深度预测数据集 SeasonDepth（可在 https://seasondepth.github.io 查阅）。为了在不同环境下评估深度估计性能的基准，我们采用数种新构建的指标，对 KITTI 基准中的代表性最新开源监督、自监督和领域自适应深度预测方法进行了研究。通过对所提数据集的广泛实验评估，我们分析了多种环境对性能均值和方差的影响，表明长期单目深度预测问题尚未得到解决。我们进一步提出了一些有前景的解决方案，尤其是通过自监督立体几何和多任务训练来增强对变化环境的鲁棒性。