THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

Generating 3D city models rapidly is crucial for many applications. Monocular height estimation is one of the most efficient and timely ways to obtain large-scale geometric information. However, existing works focus primarily on training and testing models using unbiased datasets, which don’t align well with real-world applications. Therefore, we propose a new benchmark dataset to study the transferability of height estimation models in a cross-dataset setting. To this end, we first design and construct a large-scale benchmark dataset for cross-dataset transfer learning on the height estimation task. This benchmark dataset includes a newly proposed large-scale synthetic dataset, a newly collected real-world dataset, and four existing datasets from different cities. Next, a new experimental protocol, few-shot cross-dataset transfer, is designed. For few-shot cross-dataset transfer, we enhance the window-based Transformer with the proposed scale-deformable convolution module to handle the severe scale-variation problem.

快速生成三维城市模型（3D city models）对诸多实际应用至关重要。单目高度估算是获取大规模几何信息的高效且兼具时效性的手段之一。然而，现有研究主要依托无偏数据集开展模型训练与测试，这类数据集与真实应用场景的适配性欠佳。为此，我们提出全新的基准数据集，用于研究跨数据集场景下高度估算模型的可迁移性。具体而言，我们首先设计并构建了面向高度估算任务的跨数据集迁移学习大规模基准数据集。该基准数据集包含新提出的大规模合成数据集、新采集的真实世界数据集，以及来自不同城市的四份现有数据集。随后，我们设计了全新的实验范式：少样本（Few-shot）跨数据集迁移。针对该范式下的严峻尺度变化问题，我们对基于窗口的Transformer（Transformer）模型进行增强，引入所提出的尺度可变形卷积（scale-deformable convolution）模块以解决该问题。