Robust and enhanced 360° visual tracking based on dynamic gnomonic projection

Recently, 360-degree visual tracking has become increasingly important in 360-degree video processing technology. Although visual tracking technology in 2D videos has gradually matured, there is no universal method for visual tracking in 360-degree videos that can effectively address image stretching and object deformation caused by the equirectangular representation of 360-degree images. In this paper, we propose a two-part method for 360-degree visual tracking. The first part is a general method that can be integrated into any 2D visual tracking system to be applied to 360-degree videos. This part converts equirectangular images into 2D gnomonic projections, enabling the use of existing 2D tracking algorithms while mitigating image distortion. Then, building upon the UPDT algorithm, the second part integrates the general 360-degree visual tracking method with additional enhancements to improve robustness and efficiency in 360-degree visual tracking. Furthermore, when tracking performance deteriorates, it combines results from the sample set and trajectory prediction to achieve more robust and accurate tracking. In our experiments, We use two datasets in 360-degree equirectangular representation to demonstrate the effectiveness and advantages of our proposed method. Additionally, we explore the application of 360-degree visual tracking methods in editing, enabling the automatic manipulation of moving objects in 360-degree videos.

近年来，360度视觉跟踪（360-degree visual tracking）在360度视频处理技术中的重要性与日俱增。尽管二维视频中的视觉跟踪技术已逐步趋于成熟，但目前尚无适用于360度视频的通用视觉跟踪方法，可有效解决由等距圆柱投影（equirectangular representation）导致的360度图像拉伸与目标形变问题。本文提出一种分为两部分的360度视觉跟踪方法：第一部分为通用适配方案，可集成至任意二维视觉跟踪系统以适配360度视频，该方法将等距圆柱投影图像转换为二维球心投影（gnomonic projection），从而在缓解图像畸变的同时，可直接沿用现有的二维跟踪算法。第二部分则基于UPDT算法，将上述通用360度视觉跟踪方法与额外增强模块相结合，以提升360度视觉跟踪的鲁棒性与运行效率。此外，当跟踪性能出现衰减时，该方法会结合样本集与轨迹预测的结果，实现更具鲁棒性与精准度的跟踪。在实验环节，我们采用两个采用等距圆柱投影格式的360度数据集验证了所提方法的有效性与优势。此外，我们还探索了360度视觉跟踪方法在视频编辑领域的应用，实现了360度视频中移动物体的自动化操控。