Object based Markov random field model for hierarchical semantic segmentation of remote sensing imagery

Capturing hierarchical relationships among land-cover classes is crucial for accurate semantic segmentation of remote sensing images. Traditional object-based methods face inherent limitations in modeling these complex relationships. To overcome these limitations, we proposed a novel object-based Markov random field (OMRF) model for hierarchical semantic segmentation. The objective of our model is to address two key challenges: (i) the representation of hierarchical semantic features, and (ii) the edge preservation of segmentation results. To address the first challenge, we developed hierarchical semantic representations of images for two distinct land-cover class sets and incorporated a transition probability matrix into OMRF to capture the interaction between these two semantic layers. For the second challenge, we devised an innovative spatial energy function that effectively enforces hierarchical predictions and dynamically regulates boundary smoothness by evaluating spectral dissimilarities among neighboring objects. Furthermore, a generative cross-layer inference strategy was introduced to iteratively exchange and update information across semantic layers for improved prediction. Experimental results on 11 remote sensing images demonstrate the robustness and accuracy of the proposed method, achieving an average Kappa coefficient exceeding 0.96. In comparison to 15 state-of-the-art methods, our model achieved optimal performance in 9 instances and suboptimal performance in 2 instances.