TGCNet: A Fusion Double-Branch Network with Transformer Guiding CNN for Semantic Segmentation of Remote Sensing Images

In recent years, various combined CNN and Transformer networks have achieved ideal results in the field of remote sensing image semantic segmentation. Such networks often use CNN and Transformer to obtain local details and global dependencies of the image, respectively. In order to further make the model pay more attention to the local features of remote sensing images, and solve the problems such as omission and wrong recognition caused by the difficulty of edge information extraction in the semantic segmentation task of remote sensing images, in this paper, a fusion double-branch network with Transformer giding CNN (TGCNet) is proposed. The whole frame of the model adopts encoder-decoder structure. In the encoder, Transformer branch in addition to normal coding, the global information acquired by Transformer is stratifiedly fused with the local spatial details corresponding to each layer for feature fusion, and input to the next layer of CNN coding. In addition, there is a slight difference between the feature output of Transformer and that of CNN, and feature loss may occur when feature fusion is carried out between the two outputs. In order to reduce or eliminate such feature difference, we adopt the idea of making the structure of the two branches as consistent as possible. Taking NCB block as the CNN branch of the model solves this problem well. In order to prove the effectiveness of TGCNet in semantic segmentation of remote sensing images, we compared TGCNet with seven other classical semantic segmentation models on WHU building dataset and CHN6-CUG roads dataset.The experimental results show that the mIoU of TGCNet on the two datasets is 90.23% and 82.88%, respectively, which is better than the other seven models. At The same time, the parametric quantities of TGCNet is 4.76M and FLOPs is 16.71G, better than any other network except SETR.