PROX

为了理解和分析人类行为，我们需要捕捉人类进入世界并与之互动。大多数现有方法在不明确考虑场景的情况下执行3D人体姿势估计。然而，我们观察到，世界限制了身体，反之亦然。为了激发这一点，我们表明当前的3D人体姿势估计方法产生的结果与3D场景不一致。我们的主要贡献是利用静态3D场景结构来更好地从单眼图像中估计人类姿势。该方法通过对象排除来强制执行近端关系，称为PROX。为了对此进行测试，我们收集了一个新的数据集，该数据集由12个不同的3D场景和20个移动场景并与场景交互的对象的RGB序列组成。我们使用3D人体模型SMPL-X表示人体姿势，并扩展SMPLify-X以使用场景约束来估计人体姿势。我们通过制定两个主要约束条件来利用3D场景信息。穿透约束会惩罚身体模型与周围3D场景之间的相交。如果身体的特定部位在距离和方向上足够近，则接触约束会鼓励它们与场景表面接触。为了进行定量评估，我们捕获了具有180 RGB帧的单独数据集，其中使用运动捕获系统估计了地面真相身体姿势。我们定量地表明，引入场景约束可以显着降低3D关节误差和顶点误差。

To understand and analyze human behavior, it is necessary to capture how humans interact with the world. Most existing 3D human pose estimation methods perform the task without explicitly considering the scene context. However, we observe that the physical world constrains the human body, and vice versa. To substantiate this observation, we demonstrate that current 3D human pose estimation models generate results inconsistent with the 3D scene geometry. Our primary contribution is leveraging static 3D scene structure to improve human pose estimation from monocular images. This approach enforces proximal relationships via object exclusion, termed PROX.

To validate this method, we collect a novel dataset comprising RGB sequences from 12 distinct 3D scenes, along with 20 moving objects that interact with the scenes. We adopt the SMPL-X 3D human body model to represent human poses, and extend SMPLify-X to estimate human poses by incorporating scene constraints. We utilize 3D scene information through two core constraints: the penetration penalty punishes intersections between the body model and the surrounding 3D scene, while the contact constraint encourages specific body parts to make contact with scene surfaces when they are sufficiently close in both distance and orientation.

For quantitative evaluation, we capture a separate dataset with 180 RGB frames, where ground-truth human body poses are estimated via a motion capture system. We quantitatively verify that introducing scene constraints significantly reduces both 3D joint error and vertex error.

搜集汇总

背景与挑战

背景概述

PROX数据集是一个用于3D人体姿势估计的数据集，包含12个3D场景和20个移动场景的RGB序列，使用SMPL-X模型和场景约束来提高姿势估计的准确性。数据集还提供了180个RGB帧的单独数据集用于定量评估。