khang123452/GR1-Manipulation-IsaacSim-Rendered

---
license: cc-by-4.0
task_categories:
- robotics
- video-classification
tags:
- robotics
- manipulation
- sim-to-real
- cosmos-transfer
- isaac-sim
- gr1-humanoid
- pick-and-place
pretty_name: GR1 Manipulation - Isaac Sim Rendered (RGB/Depth/Seg)
size_categories:
- 10K<n<100K
---

# GR1 Manipulation — Isaac Sim Rendered Dataset

## Dataset Description

Multi-modal rendered dataset of **NVIDIA GR1 humanoid robot** performing tabletop manipulation tasks
in Isaac Sim. Each episode is rendered as 3 synchronized MP4 videos:

- **RGB** — ray-traced color video
- **Depth** — normalized depth map (grayscale, 0–10m range)
- **Segmentation** — semantic segmentation with per-object class coloring

This dataset is designed as input for **NVIDIA Cosmos Transfer 2.5** to generate photorealistic
sim-to-real augmented training data for robot learning.

## Dataset Structure

```
{task_name}/
episode_NNNN_rgb.mp4 # 1280x720, 20fps, RayTracedLighting
episode_NNNN_depth.mp4 # 1280x720, 20fps, normalized depth
episode_NNNN_seg.mp4 # 1280x720, 20fps, semantic segmentation
```

## Tasks

| # | Task Name | Episodes | Frames/Episode |
|---|-----------|----------|----------------|
| 1 | `PnPBottleToCabinetClose` | 458 | ~120–400 |
| 2 | `PnPCanToDrawerClose` | 2 | ~120–400 |
| 3 | `PnPCupToDrawerClose` | 2 | ~120–400 |
| 4 | `PnPMilkToMicrowaveClose` | 2 | ~120–400 |
| 5 | `PnPPotatoToMicrowaveClose` | 2 | ~120–400 |
| 6 | `PnPWineToCabinetClose` | 2 | ~120–400 |
| 7 | `PosttrainPnPNovelFromCuttingboardToBasketSplitA` | 2 | ~120–400 |
| 8 | `PosttrainPnPNovelFromCuttingboardToCardboardboxSplitA` | 2 | ~120–400 |
| 9 | `PosttrainPnPNovelFromCuttingboardToPanSplitA` | 2 | ~120–400 |
| 10 | `PosttrainPnPNovelFromCuttingboardToPotSplitA` | 2 | ~120–400 |
| 11 | `PosttrainPnPNovelFromCuttingboardToTieredbasketSplitA` | 2 | ~120–400 |
| 12 | `PosttrainPnPNovelFromPlacematToBasketSplitA` | 2 | ~120–400 |
| 13 | `PosttrainPnPNovelFromPlacematToBowlSplitA` | 1 | ~120–400 |
| 14 | `PosttrainPnPNovelFromPlacematToPlateSplitA` | 2 | ~120–400 |
| 15 | `PosttrainPnPNovelFromPlacematToTieredshelfSplitA` | 2 | ~120–400 |
| 16 | `PosttrainPnPNovelFromPlateToBowlSplitA` | 2 | ~120–400 |
| 17 | `PosttrainPnPNovelFromPlateToCardboardboxSplitA` | 1 | ~120–400 |
| 18 | `PosttrainPnPNovelFromPlateToPanSplitA` | 2 | ~120–400 |
| 19 | `PosttrainPnPNovelFromPlateToPlateSplitA` | 1 | ~120–400 |
| 20 | `PosttrainPnPNovelFromTrayToCardboardboxSplitA` | 2 | ~120–400 |
| 21 | `PosttrainPnPNovelFromTrayToPlateSplitA` | 2 | ~120–400 |
| 22 | `PosttrainPnPNovelFromTrayToPotSplitA` | 2 | ~120–400 |
| 23 | `PosttrainPnPNovelFromTrayToTieredbasketSplitA` | 1 | ~120–400 |
| 24 | `PosttrainPnPNovelFromTrayToTieredshelfSplitA` | 2 | ~120–400 |

## Recording Settings

| Setting | Value |
|---------|-------|
| **Camera** | `egoview` (1st-person, attached to `robot0_head_pitch`) |
| **FOV** | 90° vertical |
| **Resolution** | 1280×720 |
| **FPS** | 20 |
| **Renderer** | Isaac Sim RayTracedLighting, SPP=32 |
| **Lighting** | Dome light, intensity=1500 |
| **USD Export** | robosuite exporter, geomgroup=[0,1,0,0,0,0] (visual meshes only) |

## Source Data

- **Robot**: NVIDIA GR1 humanoid (from gr00trobosuite)
- **Tasks**: RoboCasa GR1 tabletop pick-and-place tasks
- **Simulation**: MuJoCo → USD → Isaac Sim Replicator pipeline
- **Original trajectories**: 1000 episodes per task in HDF5 format

## Intended Use

Input for Cosmos Transfer 2.5 world generation model to produce photorealistic augmented
manipulation videos for robot policy training (sim-to-real transfer).

## License

CC-BY-4.0