sr-stereo

Due to the difficulty in obtaining real samples and ground truth, the generalization performance and the fine-tuned performance are critical for the feasibility of stereo matching methods in real-world applications. However, the diverse datasets exhibit substantial discrepancies in disparity distribution and density, thus presenting a formidable challenge to the generalization and fine-tuning of the model. In this paper, we propose a novel stereo matching method, called SR-Stereo, which mitigates the distributional differences across different datasets by predicting the disparity clips and uses a loss weight related to the regression target scale to improve the accuracy of the disparity clips. Moreover, this stepwise regression architecture can be easily extended to existing iteration-based methods to improve the performance without changing the structure. In addition, to mitigate the edge blurring of the fine-tuned model on sparse ground truth, we propose Domain Adaptation Based on Pre-trained Edges (DAPE). Specifically, we use the predicted disparity and RGB image to estimate the edge map of the target domain image. The edge map is filtered to generate edge map background pseudo-labels, which together with the sparse ground truth disparity on the target domain are used as a supervision to jointly fine-tune the pre-trained stereo matching model. These proposed methods are extensively evaluated on SceneFlow, KITTI, Middbury 2014 and ETH3D. The SR-Stereo achieves competitive disparity estimation performance and state-of-the-art cross-domain generalisation performance. Meanwhile, the proposed DAPE significantly improves the disparity estimation performance of fine-tuned models, especially in the textureless and detail regions.

鉴于获取真实样本和真实值的困难，泛化性能和微调性能对于立体匹配方法在实际应用中的可行性至关重要。然而，多样化的数据集在视差分布和密度上表现出显著的差异，这为模型的泛化和微调带来了巨大的挑战。在本文中，我们提出了一种新型的立体匹配方法，称为SR-Stereo，该方法通过预测视差片段来缓解不同数据集间的分布差异，并利用与回归目标尺度相关的损失权重来提高视差片段的准确性。此外，这种逐步回归架构可以轻松扩展到现有的基于迭代的现有方法中，以提升性能而无需改变其结构。此外，为了减轻在稀疏真实值上微调模型的边缘模糊问题，我们提出了基于预训练边缘的域自适应方法（DAPE）。具体而言，我们使用预测的视差和RGB图像来估计目标域图像的边缘图。边缘图经过滤波以生成边缘图背景伪标签，这些伪标签与目标域上的稀疏真实值视差共同用作监督，以联合微调预训练的立体匹配模型。这些提出的方法在SceneFlow、KITTI、Middbury 2014和ETH3D上进行了广泛的评估。SR-Stereo在视差估计性能和跨域泛化性能方面取得了具有竞争力的成果，同时，提出的DAPE显著提高了微调模型的视差估计性能，尤其是在无纹理和细节区域。