Homography Estimation with Adaptive Query Transformer and Gated Interaction Module

Deep models leverage long-range contexts to comprehend overall geometry, which is crucial for homography estimation. Cross-attention is commonly used to capture these long-range contexts.However, we observe that queries in weakly textured areas may introduce ambiguity during the cross-attention. Additionally, significant geometric disparities exist in image pairs used for homography estimation, resulting in numerous non-overlapping regions. Queries within the non-overlapping areas pose challenges in establishing correspondences with the counterpart image.We coin the term \textit{query overfocusing} to characterize this phenomenon.To alleviate the query overfocusing issue, this paper introduces an adaptive query transformer termed AQFormer. Specifically, we design a deep module to generate a mask based on the input of cross-attention. AQFormer suppresses the queries located in redundant and weakly textured areas and promotes the queries located in richly textured areas by the mask.In addition, we introduce a Gated Interaction Module (GIM) for local refinement. GIM encodes convolutional kernels employing a gating mechanism to extract common features from the image pairs. Extensive experiments demonstrate that the proposed network achieves state-of-the-art performance on five benchmark datasets.