HIU-DMTL-Data (Hand Image Understanding via Deep Multi-Task Learning.)

分析和理解来自多媒体材料（如图像或视频）的手部信息对于许多现实世界的应用非常重要，并且在研究界仍然很活跃。有很多工作专注于从单个图像中恢复手部信息，但是它们通常解决单个任务，例如手部蒙版分割、2D/3D 手部姿势估计或手部网格重建，并且在具有挑战性的场景中表现不佳。为了进一步提高这些任务的性能，我们提出了一种新颖的手部图像理解 (HIU) 框架，通过联合考虑这些任务之间的关系，从单个 RGB 图像中提取手部对象的综合信息。为了实现这一目标，设计了一个级联多任务学习 (MTL) 主干来估计 2D 热图，学习分割掩码，并生成中间 3D 信息编码，然后是粗到细的学习范式和自-监督学习策略

Analyzing and understanding hand-related information from multimedia materials such as images or videos is critical for numerous real-world applications and remains an active research topic. Numerous studies have focused on recovering hand information from single images, but most typically address only a single task, such as hand mask segmentation, 2D/3D hand pose estimation, or hand mesh reconstruction, and perform poorly in challenging scenarios. To further improve the performance of these tasks, we propose a novel Hand Image Understanding (HIU) framework that extracts comprehensive information of hand objects from a single RGB image by jointly considering the relationships between these tasks. To achieve this goal, a cascaded multi-task learning (MTL) backbone is designed to estimate 2D heatmaps, learn segmentation masks, and generate intermediate 3D information encodings, followed by a coarse-to-fine learning paradigm and a self-supervised learning strategy.

搜集汇总

背景与挑战

背景概述

HIU-DMTL-Data 是一个用于手部图像理解的数据集，专注于从单个RGB图像中提取手部的综合信息，包括分割、姿势估计和网格重建。它采用级联多任务学习框架和自监督学习策略，以提升在挑战性场景中的性能。该数据集由百度、快手等机构于2021年发布。