SeasonDepth: Cross-Season Monocular Depth Prediction Training Dataset

<pre>Changing environments poses great challenge on the outdoor visual perception and scene understanding for robust long-term autonomous driving and mobile robots, where depth-auxiliary geometric information plays an essential role to the robustness under challenging scenes. Although monocular depth prediction has been well studied recently, there are few work focused on the depth prediction across multiple environmental conditions, e.g. changing illumination and seasons, owing to the lack of such real-world dataset and benchmark. In this work, a new cross-season scaleless monocular depth prediction dataset SeasonDepth (Available on https://seasondepth.github.io) is derived from CMU Visual Localization dataset through structure from motion. To benchmark the depth estimation performance under different environments, we investigate representative and recent state-of-the-art open-source supervised, self-supervised and domain adaptation depth prediction methods from KITTI benchmark using several newly-formulated metrics. Through extensive experimental evaluation on the proposed dataset without fine-tuning, the influence of multiple environments on mean and variance of performance is analyzed, showing that the long-term monocular depth prediction is far from solved. We further give promising solutions especially with self-supervised stereo geometry and multi-task training to enhance the robustness to changing environments. </pre>

不断变化的环境为鲁棒的长期自动驾驶与移动机器人的室外视觉感知与场景理解带来了巨大挑战，而深度辅助几何信息在复杂场景下对提升系统鲁棒性起到至关重要的作用。尽管近期单目深度预测（monocular depth prediction）已得到广泛研究，但由于缺乏对应的真实世界数据集与基准测试集，目前鲜有面向多环境条件（如光照与季节变化）下深度预测的研究工作。本工作通过运动恢复结构（Structure from Motion，SfM）从CMU视觉定位数据集（CMU Visual Localization dataset）中构建了一个全新的跨季节无尺度单目深度预测数据集SeasonDepth，相关资源可通过https://seasondepth.github.io获取。为构建不同环境下的深度估计性能基准，我们基于KITTI基准测试集（KITTI benchmark），采用多种新设计的评估指标，对当前具有代表性的前沿开源监督式（supervised）、自监督式（self-supervised）与域自适应（domain adaptation）深度预测方法进行了调研与测试。通过在所提出的数据集上开展无需微调的大规模实验评估，我们分析了多环境因素对模型性能均值与方差的影响，结果表明长期单目深度预测问题仍远未得到解决。我们进一步提出了颇具前景的解决方案，尤其是结合自监督立体几何与多任务训练的方法，以提升模型对环境变化的鲁棒性。