THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

Generating 3D city models rapidly is crucial for many applications. Monocular height estimation is one of the most efficient and timely ways to obtain large-scale geometric information. However, existing works focus primarily on training and testing models using unbiased datasets, which don’t align well with real-world applications. Therefore, we propose a new benchmark dataset to study the transferability of height estimation models in a cross-dataset setting. To this end, we first design and construct a large-scale benchmark dataset for cross-dataset transfer learning on the height estimation task. This benchmark dataset includes a newly proposed large-scale synthetic dataset, a newly collected real-world dataset, and four existing datasets from different cities. Next, a new experimental protocol, few-shot cross-dataset transfer, is designed. For few-shot cross-dataset transfer, we enhance the window-based Transformer with the proposed scale-deformable convolution module to handle the severe scale-variation problem.快速生成三维城市模型对诸多应用场景而言至关重要。单目高度估计是获取大规模几何信息的最为高效及时的途径之一。然而，现有研究主要基于无偏数据集开展模型训练与测试，这类数据集与实际应用场景适配性不佳。为此，我们提出全新的基准数据集，用于研究跨数据集场景下高度估计模型的迁移性能。具体而言，我们首先针对高度估计任务的跨数据集迁移学习，设计并构建了大规模基准数据集。该基准数据集包含全新提出的大规模合成数据集、新采集的真实世界数据集，以及来自不同城市的四份现有数据集。随后，我们设计了全新的少样本跨数据集迁移实验范式。针对少样本跨数据集迁移任务，我们将所提出的尺度可变形卷积模块集成至基于窗口的Transformer中，以解决严重的尺度变化问题。