GRAB

训练计算机理解，建模和合成人类抓握需要一个丰富的数据集，其中包含复杂的3D对象形状，详细的联系信息，手的姿势和形状以及随时间变化的3D身体运动。虽然 “抓握” 通常被认为是单手稳定地举起物体，但我们捕捉整个身体的运动，并采用 “全身抓握” 的广义概念。因此，我们收集了一个新的数据集，称为抓握 (用身体抓握动作)，全身抓握，包含10个受试者与51个不同形状和大小的日常物体相互作用的完整3D形状和姿势序列。给定MoCap标记，我们适合完整的3D身体形状和姿势，包括关节的面部和手，以及3D对象姿势。随着时间的推移，这给出了详细的3D网格，我们从中计算身体和物体之间的接触。这是一个独特的数据集，它远远超出了现有的数据集，用于建模和理解人类如何抓住和操纵对象，他们的全身如何参与，以及交互如何随着任务而变化。我们通过示例应用程序说明了抓取的实用价值; 我们训练有条件生成网络GrabNet，以预测看不见的3D对象形状的3D手抓取。

Training computers to understand, model, and synthesize human grasping requires a rich dataset containing complex 3D object shapes, detailed contact information, hand poses and shapes, as well as 3D bodily motions over time. While "grasping" is often defined as the stable lifting of an object with a single hand, we capture full-body motions and adopt a generalized concept of "whole-body grasping". Therefore, we collect a novel dataset named Grasp (Grasping Actions with the Body): Whole-Body Grasping, which contains complete 3D shape and pose sequences of 10 human subjects interacting with 51 distinct everyday objects of varying shapes and sizes. Given the MoCap markers, we fit complete 3D body shapes and poses, including the articulated faces and hands, as well as 3D object poses. Over time, this yields detailed 3D meshes, from which we calculate the contact between the body and the objects. This is a unique dataset that far surpasses existing datasets for modeling and understanding how humans grasp and manipulate objects, how their full bodies are involved in the interaction, and how the interaction evolves across different tasks. We demonstrate the practical value of Grasp via exemplary applications; we train a conditional generative network named GrabNet to predict 3D hand grasps for previously unseen 3D object shapes.

搜集汇总

背景与挑战

背景概述

GRAB是一个专注于人类全身抓握动作的3D数据集，包含10名受试者与51个日常物体的交互数据，提供了详细的3D形状、姿势序列和接触信息，由马克斯普朗克智能系统研究所于2020年发布。