SensoriRobotics/g1_locomanipulation_sdg

---
tags:
- lerobot
- robotics
- humanoid
- unitree-g1
- vla
- gr00t
- imitation-learning
- locomanipulation
license: bsd-3-clause
---

# G1 Locomanipulation SDG

Synthetic visuomotor dataset for **Unitree G1** locomanipulation (pick-and-place with point-to-point navigation), generated from Isaac Lab's locomanipulation-SDG pipeline and converted to LeRobot v2.0 format for **GR00T N1.6** finetuning.

## Dataset summary

- **55 episodes**, ~58k frames total
- **20 fps**
- **Ego-view camera**: `observation.images.ego_view`, 160×256×3
- **Task prompt**: `"Pick up and drop off the object"`

## State (49 dims)

| Key | Range | Description |
|---|---|---|
| `left_hand_pose` | 0:7 | Left wrist pose (pos + quat, base-relative) |
| `right_hand_pose` | 7:14 | Right wrist pose (pos + quat, base-relative) |
| `left_hand_joint_positions` | 14:21 | 7-DOF left hand joints |
| `right_hand_joint_positions` | 21:28 | 7-DOF right hand joints |
| `object_pose` | 28:35 | Object pose (base-relative) |
| `goal_pose` | 35:42 | Base goal pose (base-relative) |
| `end_fixture_pose` | 42:49 | Drop-off fixture pose (base-relative) |

## Action

Hand poses + hand joint positions + base velocity + base height (exact slicing in `meta/modality.json`).

## Generation pipeline

1. Teleoperation recording via Quest controllers (Isaac Lab Mimic)
2. Manual subtask annotation
3. Isaac Lab Mimic dataset generation (state-only, ~1000 source demos)
4. Locomanipulation SDG with randomized obstacle/fixture placement and AGILE locomotion policy
5. LeRobot v2.0 conversion (this dataset)

See the [Isaac Lab imitation-learning docs](https://isaac-sim.github.io/IsaacLab/develop/source/overview/imitation-learning/mimic_humanoid_demos.html) for the full pipeline.

## Intended use

Finetuning **NVIDIA GR00T N1.5/N1.6** for G1 locomanipulation. Training is driven by the `g1_locomanipulation_sdg` entry in GR00T's `DATA_CONFIG_MAP` (see `scripts/imitation_learning/locomanipulation_sdg/gr00t/data_config.py` in Isaac Lab).

## Visualize

- **Local**: `python -m lerobot.scripts.visualize_dataset --repo-id SensoriRobotics/g1_locomanipulation_sdg --root <local_path> --local-files-only --episode-index 0`
- **HF Space**: [lerobot/visualize_dataset](https://huggingface.co/spaces/lerobot/visualize_dataset)

---
标签：
- LeRobot
- 机器人学
- 人形机器人
- 宇树G1（Unitree G1）
- 视觉语言动作模型（VLA）
- GR00T
- 模仿学习（imitation-learning）
- 移动操作（locomanipulation）
许可证：BSD-3-Clause
---

# G1 移动操作合成数据集（G1 Locomanipulation SDG）
本数据集为面向**宇树G1（Unitree G1）**的移动操作任务（带点对点导航的拾取放置任务）合成视觉运动数据集，由Isaac Lab的移动操作合成数据生成流水线生成，并转换为LeRobot v2.0格式，用于**GR00T N1.6**的微调。

## 数据集概览
- **共55个回合**，总帧数约5.8万
- **帧率为20 fps**
- **第一视角摄像头**：`observation.images.ego_view`，分辨率为160×256×3
- **任务提示**："拾取并放置目标物体"

## 状态信息（49维）
| 键名 | 维度范围 | 描述 |
|---|---|---|
| `left_hand_pose` | 0:7 | 左手腕位姿（位置+四元数，相对于基座） |
| `right_hand_pose` | 7:14 | 右手腕位姿（位置+四元数，相对于基座） |
| `left_hand_joint_positions` | 14:21 | 7自由度左手关节位置 |
| `right_hand_joint_positions` | 21:28 | 7自由度右手关节位置 |
| `object_pose` | 28:35 | 目标物体位姿（相对于基座） |
| `goal_pose` | 35:42 | 基座导航目标位姿（相对于基座） |
| `end_fixture_pose` | 42:49 | 放置夹具位姿（相对于基座） |

## 动作空间
包含手部位姿、手部关节位置、基座线速度及基座高度，具体维度切片详见`meta/modality.json`。

## 数据生成流水线
1. 通过Quest控制器（Isaac Lab Mimic）进行遥操作录制
2. 手动标注子任务
3. 生成仅含状态信息的Isaac Lab Mimic数据集（约1000个源演示样本）
4. 采用随机化障碍物/放置夹具布局与通用人工智能（AGI，Artificial General Intelligence）运动策略，执行移动操作合成数据生成
5. 转换为LeRobot v2.0格式，即本数据集

可参考[Isaac Lab模仿学习文档](https://isaac-sim.github.io/IsaacLab/develop/source/overview/imitation-learning/mimic_humanoid_demos.html)查看完整流水线。

## 预期用途
本数据集用于面向宇树G1的移动操作任务，微调**NVIDIA GR00T N1.5/N1.6**模型。训练可通过GR00T的`DATA_CONFIG_MAP`中`g1_locomanipulation_sdg`条目启动，详见Isaac Lab中的`scripts/imitation_learning/locomanipulation_sdg/gr00t/data_config.py`文件。

## 数据集可视化
- **本地可视化命令**：`python -m lerobot.scripts.visualize_dataset --repo-id SensoriRobotics/g1_locomanipulation_sdg --root <本地路径> --local-files-only --episode-index 0`
- **Hugging Face Space可视化工具**：[lerobot/visualize_dataset](https://huggingface.co/spaces/lerobot/visualize_dataset)