MOWA: Multiple-in-One Image Warping Model

While recent image warping approaches achieved remarkable success on existing benchmarks, they still require training separate models for each specific task and cannot generalize well to different camera models or customized manipulations. To address diverse types of warping in practice, we propose a Multiple-in-One image WArping model (named MOWA) in this work. Specifically, we mitigate the difficulty of multi-task learning by disentangling the motion estimation at both the region level and pixel level. To further enable dynamic task-aware image warping, we introduce a lightweight point-based classifier that predicts the task type, serving as prompts to modulate the feature maps for more accurate estimation. To our knowledge, this is the first work that solves multiple practical warping tasks in one single model. Extensive experiments demonstrate that our MOWA, which is trained on six tasks for multiple-in-one image warping, outperforms state-of-the-art task-specific models across most tasks. Moreover, MOWA also exhibits promising potential to generalize into unseen scenes, as evidenced by cross-domain and zero-shot evaluations.

尽管近期的图像扭曲（image warping）方法在现有基准测试中已取得显著成果，但此类方法仍需针对每项特定任务单独训练模型，且难以泛化适配不同相机模型或自定义操作。为满足实际场景中多样化的图像扭曲任务需求，本文提出一款多合一图像扭曲模型（Multiple-in-One image WArping，简称MOWA）。具体而言，我们通过在区域级与像素级两个维度解耦运动估计任务，降低了多任务学习的训练难度。为进一步实现动态任务感知的图像扭曲，我们引入一款轻量级基于点的分类器，用于预测任务类型，并将其作为提示信号调制特征图，以获得更精准的估计结果。据我们所知，本研究是首个在单一模型中完成多种实用图像扭曲任务的工作。大量实验结果表明，在六项多合一图像扭曲任务上训练得到的MOWA模型，在绝大多数任务上均优于当前顶尖的单任务专用模型。此外，跨域与零样本（zero-shot）评估结果证实，MOWA还展现出良好的泛化至未知场景的潜力。