habitat_humanoids

# Habitat Humanoids


Habitat 3.0 provides support for diverse humanoid avatars, displaying different shapes an motions. Avatars are based on the [SMPL-X](https://smpl-x.is.tue.mpg.de/) body model format, a commonly used data-driven parametric human body model that provides a compact representation of 3D human shape and pose.

This repository provides a set of stand-alone avatars and motion files to represent humanoids walking and reaching to objects in the Habitat simulator. However, you can also generate new humanoids using the SMPL-X code base, or use motions coming from motion capture or motion generation models.

## Contents

We provide a total of 12 textured avatars of neutral, female and male gender and covering different body shapes. For each avatar, we provide a motion file that allows to drive the avatar to walk in a scene, or reach out to objects, using a [controller](https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/articulated_agent_controllers/humanoid_rearrange_controller.py).

The folder structure is as follows:

```
├── habitat_humanoids
│ ├── neutral_0
| | ├── neutral_0.ao_config.json
| | ├── neutral_0.glb
| | ├── neutral_0_motion_data_smplx.pkl
| | ├── neutral_0.urdf
│ ├── *
│ ├── walk_motion
│ | ├── CMU_10_04_stageii.npz
```

Where neutral_0 corresponds to the folder of one of the textured avatars.

- `neutral_0.ao_config.json`: contains a dictionary with information on how to link the avatar armature and skinning, and the semantic id of the avatar, when using a semantic sensor.
- `neutral_0.glb`: contains the skinning and texture information.
- `neutral_0_motion_data_smplx.pkl`: contains relevant motion data files, more information below.
- `neutral_0.urdf`: contains the armature, built automatically from the SMPL-X body model.
- `walk_motion/CMU_10_04_stageii.npz`: contains a clip file from AMASS, used to build our motion file.

### Motion Data File

For each avatar, we provide a dictionary stored in `*_motion_data_smplx.pkl` which contains information to animate the character to walk around a scene and reach out to different positions.In particular, the dictionary contains 3 keys to store this information.

- `walk_motion`: contains a 130 frame clip of a person performing a walking cycle. In particular, the clip corresponds to the frames 300-430 of the file `CMU/10/10_04_stageii.npz` from AMASS dataset. We provide the raw data in this repository, released under a license detailed below.
- `stop_pose`: contains a standing position, taken from s single frame from the motion clip above mentioned.
- `left_hand`: Contains a grid of poses 48 generated using [VPoser](https://github.com/nghorbani/human_body_prior), where each pose is optimized to reach a given poisition in 3D. In [HumanoidRearrangeController](https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/articulated_agent_controllers/humanoid_rearrange_controller.py), we provide code to interpolate over these poses to reach multiple 3D positions.
- `right_hand`: Contains the same grid of poses to reach positions with the *right hand*.

## Usage

Clone this file under `data/`. We provide several files in the [habitat-lab repository](https://github.com/facebookresearch/habitat-lab) to instantiate and move the avatars around the scene.

## License

The 12 provided avatars, along with their textures, and the reaching positions stored in `left_hand` and `right_hand` are released under a [CC-BY-NC 4.0 License](https://creativecommons.org/licenses/by-nc/4.0/deed.en).

The motion data stored in `walk_motion` and `stop_pose`, as well as the original file `CMU_10_04_stageii.npz` is released under the [SMPL Body Motion File License](https://smpl.is.tue.mpg.de/bodylicense.html), a Creative Commons Attribution 4.0 International License. For support or inquiries about more SMPL Body Mo.on Files for commercial use, please contact [e-mail].
git

# Habitat 类人机器人（Habitat Humanoids）


Habitat 3.0 支持多样化的类人化身，可呈现各异的身形与动作。此类化身基于[SMPL-X（SMPL-X）](https://smpl-x.is.tue.mpg.de/)人体模型格式构建——这是一种广泛使用的数据驱动型参数化人体模型，可紧凑表征三维人体的身形与姿态。

本仓库提供了一系列独立的类人化身与动作文件，用于在Habitat模拟器中呈现类人机器人行走及触碰物体的行为。此外，用户也可基于SMPL-X代码库生成全新的类人化身，或直接使用动作捕捉数据、动作生成模型输出的动作序列。

## 内容说明

本次数据集共提供12个带纹理的类人化身，涵盖中性、女性、男性三种性别，且身形各异。针对每个化身，我们均配套提供了动作文件，可通过[控制器（controller）](https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/articulated_agent_controllers/humanoid_rearrange_controller.py)控制化身在场景中行走或触碰物体。

文件夹结构如下：

├── habitat_humanoids
│ ├── neutral_0
| | ├── neutral_0.ao_config.json
| | ├── neutral_0.glb
| | ├── neutral_0_motion_data_smplx.pkl
| | ├── neutral_0.urdf
│ ├── *
│ ├── walk_motion
│ | ├── CMU_10_04_stageii.npz

其中`neutral_0`即为其中一个带纹理类人化身的文件夹。

- `neutral_0.ao_config.json`：存储一个字典，包含类人化身的骨骼绑定与蒙皮关联方式，以及使用语义传感器时化身的语义ID信息。
- `neutral_0.glb`：存储蒙皮与纹理信息。
- `neutral_0_motion_data_smplx.pkl`：存储相关动作数据文件，详细说明见下文。
- `neutral_0.urdf`：存储骨骼结构，该结构由SMPL-X人体模型自动生成。
- `walk_motion/CMU_10_04_stageii.npz`：存储来自AMASS数据集的片段文件，用于构建本数据集的动作文件。

### 动作数据文件

针对每个类人化身，我们将所需信息存储于`*_motion_data_smplx.pkl`字典文件中，用于驱动角色在场景中行走及触碰不同位置。该字典共包含4组键值对以存储相关信息：

- `walk_motion`：存储一段包含130帧的行走循环动作片段。具体而言，该片段取自AMASS数据集中`CMU/10/10_04_stageii.npz`文件的第300至430帧。本仓库已附带该原始数据，其授权协议详见下文。
- `stop_pose`：存储一个站立姿态，取自上述动作片段中的单帧数据。
- `left_hand`：存储由[VPoser（VPoser）](https://github.com/nghorbani/human_body_prior)生成的48个手部姿态网格，每个姿态均经过优化以适配三维空间中的指定触碰位置。在[类人机器人重排控制器（HumanoidRearrangeController）](https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/articulated_agent_controllers/humanoid_rearrange_controller.py)中，我们提供了可在这些姿态间进行插值以实现多位置触碰的代码。
- `right_hand`：存储一组与左手姿态网格完全一致的姿态，用于实现右手的位置触碰。

## 使用方法

将本仓库克隆至`data/`目录下。我们在[habitat-lab仓库（habitat-lab repository）](https://github.com/facebookresearch/habitat-lab)中提供了多个文件，用于实例化类人化身并控制其在场景中移动。

## 授权协议

本次提供的12个类人化身及其纹理，以及存储于`left_hand`和`right_hand`中的触碰位置数据，均采用[CC-BY-NC 4.0授权协议（CC-BY-NC 4.0 License）](https://creativecommons.org/licenses/by-nc/4.0/deed.en)进行发布。

存储于`walk_motion`和`stop_pose`中的动作数据，以及原始文件`CMU_10_04_stageii.npz`，则采用[SMPL人体动作文件授权协议（SMPL Body Motion File License）](https://smpl.is.tue.mpg.de/bodylicense.html)（即Creative Commons Attribution 4.0 International License）进行发布。若需获取更多SMPL人体动作文件的商业授权或相关咨询，请联系[e-mail]。