SeasonDepth: Cross-Season Monocular Depth Prediction Dataset

Changing environments poses great challenge on the outdoor visual perception and scene understanding for robust long-term autonomous driving and mobile robots, where depth-auxiliary geometric information plays an essential role to the robustness under challenging scenes. Although monocular depth prediction has been well studied recently, there are few work focused on the depth prediction across multiple environmental conditions, e.g. changing illumination and seasons, owing to the lack of such real-world dataset and benchmark. In this work, a new cross-season scaleless monocular depth prediction dataset SeasonDepth (Available on https://seasondepth.github.io) is derived from CMU Visual Localization dataset through structure from motion. To benchmark the depth estimation performance under different environments, we investigate representative and recent state-of-the-art open-source supervised, self-supervised and domain adaptation depth prediction methods from KITTI benchmark using several newly-formulated metrics. Through extensive experimental evaluation on the proposed dataset without fine-tuning, the influence of multiple environments on mean and variance of performance is analyzed, showing that the long-term monocular depth prediction is far from solved. We further give promising solutions especially with self-supervised stereo geometry and multi-task training to enhance the robustness to changing environments.不断变化的环境对鲁棒的长期自动驾驶与移动机器人的室外视觉感知与场景理解构成了巨大挑战，而深度辅助几何信息在复杂场景下对保障系统鲁棒性起到关键作用。尽管近年来单目深度预测已得到充分研究，但由于缺乏此类真实世界数据集与基准测试集，针对多环境条件（如光照与季节变化）下的深度预测研究仍鲜有开展。本研究通过运动恢复结构从CMU视觉定位数据集中构建了一款全新的跨季节无尺度单目深度预测数据集SeasonDepth，相关资源可从https://seasondepth.github.io获取。为了在不同环境下开展深度估计性能的基准测试，我们采用多种新构建的评估指标，对源自KITTI基准测试集的代表性最新开源监督式、自监督式以及域自适应深度预测方法进行了调研。通过在该数据集上开展无需微调的大规模实验评估，我们分析了多环境因素对模型性能均值与方差的影响，结果表明长期单目深度预测问题仍远未得到解决。我们进一步提出了颇具前景的解决方案，尤其是结合自监督立体几何与多任务训练的方法，以提升模型对环境变化的鲁棒性。