用于单视图高度估计和语义分割的SCE-Net自增强和交叉增强网络

近年来，单视角高度估计和语义分割受到了越来越多的关注，并在摄影测量和遥感社区中发挥了重要作用。图像的高度信息和语义信息是相关的，一些最新的研究表明，多任务学习方法可以实现任务相关特征的互补，并提高多个任务的预测结果。尽管最近的研究取得了很多进展，但如何有效地提取和融合高度特征和语义特征仍然是一个悬而未决的问题。在本文中，我们提出了一个自我和交叉增强网络（SCE-Net）来共同对单一航空图像进行高度估计和语义分割。基于注意机制，构建了一个特征分离-融合模块，用于有效地分离和融合高度特征和语义特征，以增强任务间的特征表示。此外，还设计了一个基于深度度量学习的高度引导特征距离损失和语义引导特征距离损失，以实现面向任务的特征表示增强。在Vaihingen数据集和Potsdam数据集上进行了大量实验，以验证所提方法的有效性。实验结果表明，所提出的SCE-Net可以胜过最先进的方法，并在高度估计和语义分割两方面都取得了更好的性能。

In recent years, single-view height estimation and semantic segmentation have attracted growing attention and played a vital role in the photogrammetry and remote sensing communities. The height information and semantic information of images are correlated, and several recent studies have demonstrated that multi-task learning methods can achieve complementarity of task-related features and improve the prediction performance of multiple tasks. Despite the numerous advancements achieved in recent research, how to effectively extract and fuse height features and semantic features remains an open question. In this paper, we propose a Self and Cross Enhancement Network (SCE-Net) for joint height estimation and semantic segmentation of single aerial images. Based on the attention mechanism, we construct a Feature Separation-Fusion Module to effectively separate and fuse height features and semantic features, thereby enhancing the inter-task feature representations. Furthermore, we design two task-guided feature distance losses, namely height-guided feature distance loss and semantic-guided feature distance loss, based on deep metric learning, to achieve task-oriented feature representation enhancement. Extensive experiments are conducted on the Vaihingen and Potsdam datasets to validate the effectiveness of the proposed method. The experimental results show that the proposed SCE-Net outperforms state-of-the-art methods and achieves better performance on both height estimation and semantic segmentation tasks.

搜集汇总

背景与挑战

背景概述

该数据集包含用于单视图高度估计和语义分割的SCE-Net方法的相关数据，主要涉及Vaihingen和Potsdam两个数据集，旨在通过自增强和交叉增强网络提升高度估计和语义分割的准确性。