PhysicalAI-Robotics-GraspGen

# GraspGen: Scaling Sim2Real Grasping
GraspGen is a large-scale simulated grasp dataset for multiple robot embodiments and grippers.

<img src="assets/cover.png" width="1000" height="250" title="readme1">

We release over 57 million grasps, computed for a subset of 8515 objects from the [Objaverse XL](https://objaverse.allenai.org/) (LVIS) dataset. These grasps are specific to three grippers: Franka Panda, the Robotiq-2f-140 industrial gripper, and a single-contact suction gripper (30mm radius).

<img src="assets/montage2.png" width="1000" height="500" title="readme2">

## Dataset Format
The dataset is released in the [WebDataset](https://github.com/webdataset/webdataset) format. The folder structure of the dataset is as follows:
```
grasp_data/
franka/shard_{0-7}.tar
robotiq2f140/shard_{0-7}.tar
suction/shard_{0-7}.tar
splits/
franka/{train/valid}_scenes.json
robotiq2f140/{train/valid}_scenes.json
suction/{train/valid}_scenes.json
```
We release test-train splits along with the grasp dataset. The splits are made randomly based on object instances.

Each json file in the shard has the following data in a python dictionary. Note that `num_grasps=2000` per object.
```
‘object’/
‘scale’ # This is the scale of the asset, float
‘grasps’/
‘object_in_gripper’ # boolean mask indicating grasp success, [num_grasps X 1]
‘transforms’ # Pose of the gripper in homogenous matrices, [num_grasps X 4 X 4]
```

The coordinate frame convention for the three grippers are provided below:
<img src="assets/grippers.png" width="450" height="220" title="readme3">

## Visualizing the dataset

We have provided some minimal, standalone scripts for visualizing this dataset. See the header of the [visualize_dataset.py](scripts/visualize_dataset.py) for installation instructions.

Before running any of the visualization scripts, remember to start meshcat-server in a separate terminal:
``` shell
meshcat-server
```

To visualize a single object from the dataset, alongside its grasps:
```shell
cd scripts/ && python visualize_dataset.py --dataset_path /path/to/dataset --object_uuid {object_uuid} --object_file /path/to/mesh --gripper_name {choose from: franka, suction, robotiq2f140}
```

To sequentially visualize a list of objects with its grasps:
```shell
cd scripts/ && python visualize_dataset.py --dataset_path /path/to/dataset --uuid_list {path to a splits.json file} --uuid_object_paths_file {path to a json file mapping uuid to absolute path of meshes} --gripper_name {choose from: franka, suction, robotiq2f140}
```

## Objaverse dataset
Please download the Objaverse XL (LVIS) objects separately. See the helper script [download_objaverse.py](scripts/download_objaverse.py) for instructions and usage.
Note that running this script autogenerates a file that maps from `UUID` to the asset mesh path, which you can pass in as input `uuid_object_paths_file` to the `visualize_dataset.py` script.

## License
License Copyright © 2025, NVIDIA Corporation & affiliates. All rights reserved.

Both the dataset and visualization code is released under a CC-BY 4.0 [License](LICENSE_DATASET).

For business inquiries, please submit the form [NVIDIA Research Licensing](https://www.nvidia.com/en-us/research/inquiries/).

## Contact

Please reach out to [Adithya Murali](http://adithyamurali.com) ([e-mail]) and [Clemens Eppner](https://clemense.github.io/) ([e-mail]) for further enquiries.

# GraspGen：面向Sim2Real抓取的规模化数据集
GraspGen是一款面向多机器人形态与多夹爪的大规模仿真抓取数据集。

<img src="assets/cover.png" width="1000" height="250" title="readme1">

我们发布了超过5700万条抓取数据，这些数据是针对[Objaverse XL](https://objaverse.allenai.org/)（LVIS）数据集中的8515个物体子集计算得到的。本次发布的抓取数据适配三款夹爪：Franka Panda机器人、Robotiq-2f-140工业夹爪，以及单触点吸盘夹爪（半径30mm）。

<img src="assets/montage2.png" width="1000" height="500" title="readme2">

## 数据集格式
本数据集采用[WebDataset](https://github.com/webdataset/webdataset)格式进行发布。数据集的文件夹结构如下：

grasp_data/
franka/shard_{0-7}.tar
robotiq2f140/shard_{0-7}.tar
suction/shard_{0-7}.tar
splits/
franka/{train/valid}_scenes.json
robotiq2f140/{train/valid}_scenes.json
suction/{train/valid}_scenes.json

本次发布同步提供了训练集与测试集的划分方案，划分依据为物体实例，采用随机划分方式。

每个分片中的JSON文件以Python字典格式存储以下数据。需注意，每个物体对应2000条抓取数据。

‘object’/
‘scale’ # 该值为资产的缩放比例，浮点型数值
‘grasps’/
‘object_in_gripper’ # 用于指示抓取是否成功的布尔掩码，维度为[num_grasps × 1]
‘transforms’ # 夹爪的齐次变换矩阵位姿，维度为[num_grasps × 4 × 4]

三款夹爪的坐标系约定如下：
<img src="assets/grippers.png" width="450" height="220" title="readme3">

## 数据集可视化
我们提供了若干轻量独立的脚本用于该数据集的可视化。安装说明请参阅[visualize_dataset.py](scripts/visualize_dataset.py)的文件头部注释。

在运行任意可视化脚本前，请务必在独立终端中启动meshcat-server：
shell
meshcat-server

若需可视化数据集中单个物体及其抓取数据：
shell
cd scripts/ && python visualize_dataset.py --dataset_path /path/to/dataset --object_uuid {object_uuid} --object_file /path/to/mesh --gripper_name {franka、suction、robotiq2f140 三选一}

若需按顺序可视化列表中的多个物体及其抓取数据：
shell
cd scripts/ && python visualize_dataset.py --dataset_path /path/to/dataset --uuid_list {path to a splits.json file} --uuid_object_paths_file {path to a json file mapping uuid to absolute path of meshes} --gripper_name {franka、suction、robotiq2f140 三选一}

## Objaverse数据集
请自行下载Objaverse XL（LVIS）数据集的物体资源。相关使用说明与操作指南请参阅辅助脚本[download_objaverse.py](scripts/download_objaverse.py)。

需注意，运行该脚本将自动生成一个映射文件，用于关联`UUID`与资源网格文件路径，该文件可作为`uuid_object_paths_file`参数输入至`visualize_dataset.py`脚本中。

## 授权协议
版权所有 © 2025 NVIDIA公司及其关联机构。保留所有权利。

本数据集及可视化代码均采用CC-BY 4.0 [授权协议](LICENSE_DATASET)发布。

商业合作咨询请填写[NVIDIA研究合作申请表](https://www.nvidia.com/en-us/research/inquiries/)。

## 联系方式
若有其他疑问，请联系[Adithya Murali](http://adithyamurali.com)（邮箱：[e-mail]）与[Clemens Eppner](https://clemense.github.io/)（邮箱：[e-mail]）。