rajeshramana/leisaac-pick-orange-prepared

---
language:
- en
license: apache-2.0
tags:
- robotics
- isaac-lab
- manipulation
- lerobot
- so101
- imitation-learning
task_categories:
- robotics
size_categories:
- 10K<n<100K
---

# LeIsaac PickOrange — Prepared Dataset (GR00T-ready)

Pre-processed version of [LightwheelAI/leisaac-pick-orange](https://huggingface.co/datasets/LightwheelAI/leisaac-pick-orange) ready for GR00T N1.6 fine-tuning.

## What's Different from the Original?

The original dataset requires several modifications to work with GR00T N1.6. This version has all fixes pre-applied:

| Fix | Original | This Dataset |
|-----|----------|-------------|
| Video codec | AV1 | **H.264** (decord-compatible) |
| Annotation column | Missing | **Integer task_index** added to all parquet files |
| modality.json | Missing | **Included** (state/action/video/annotation) |
| Format | LeRobot v2.1 | LeRobot v2.1 (compatible) |

## Dataset Details

- **Task**: LeIsaac-SO101-PickOrange-v0 (kitchen scene, pick 3 oranges → plate)
- **Robot**: SO-101 follower (5 arm joints + 1 gripper)
- **Episodes**: 60 teleoperation demonstrations
- **Frames**: ~36,000 total
- **FPS**: 30
- **Cameras**: Front (480x640) + Wrist (480x640)
- **Actions**: 6D absolute joint positions in degrees

### Action Space

```
Index | Joint | Range (deg)
------|----------------|-------------
0 | shoulder_pan | [-38, 52]
1 | shoulder_lift | [-100, 64]
2 | elbow_flex | [-99, 99]
3 | wrist_flex | [22, 100]
4 | wrist_roll | [-14, 51]
5 | gripper | [1, 91]
```

### modality.json

```json
{
"state": {
"single_arm": {"start": 0, "end": 5},
"gripper": {"start": 5, "end": 6}
},
"action": {
"single_arm": {"start": 0, "end": 5},
"gripper": {"start": 5, "end": 6}
},
"video": {
"front": {"original_key": "observation.images.front"},
"wrist": {"original_key": "observation.images.wrist"}
},
"annotation": {
"human.task_description": {"original_key": "annotation.human.task_description"}
}
}
```

## Usage with GR00T

```bash
# 1. Clone GR00T
git clone https://github.com/NVIDIA/Isaac-GR00T.git

# 2. Download this dataset
huggingface-cli download rajeshramana/leisaac-pick-orange-prepared \
--local-dir ./demo_data/pick_orange --repo-type dataset

# 3. Fine-tune
python gr00t/experiment/launch_finetune.py \
--base-model-path nvidia/GR00T-N1.6-3B \
--dataset-path ./demo_data/pick_orange \
--modality-config-path ./so101_pick_orange_config.py \
--embodiment-tag NEW_EMBODIMENT \
--num-gpus 1 \
--max-steps 10000 \
--no-tune-diffusion-model
```

## Trained Model

A model fine-tuned on this dataset for 10K steps (loss 0.017) is available at:
[rajeshramana/groot-n1.6-pick-orange](https://huggingface.co/rajeshramana/groot-n1.6-pick-orange)

## Pre-processing Steps Applied

1. **Re-encoded all 120 videos** from AV1 → H.264:
```bash
ffmpeg -y -i input.mp4 -c:v libx264 -crf 23 -preset fast -pix_fmt yuv420p output.mp4
```

2. **Added annotation column** to all 60 episode parquet files:
```python
df["annotation.human.task_description"] = 0 # integer task_index
```

3. **Created modality.json** mapping flat columns to GR00T's expected modality structure

## Source

- **Original dataset**: [LightwheelAI/leisaac-pick-orange](https://huggingface.co/datasets/LightwheelAI/leisaac-pick-orange)
- **Framework**: [LightwheelAI/leisaac](https://github.com/LightwheelAI/leisaac)
- **Simulation**: NVIDIA Isaac Lab 2.3.2

## Citation

```bibtex
@misc{leisaac-pick-orange-prepared,
title={LeIsaac PickOrange Prepared Dataset for GR00T},
author={Rajesh Kumar},
year={2026},
url={https://huggingface.co/datasets/rajeshramana/leisaac-pick-orange-prepared}
}
```

语言：
- en
许可证：apache-2.0
标签：
- 机器人学
- Isaac Lab
- 操作任务
- LeRobot
- SO101
- 模仿学习（imitation learning）
任务类别：
- 机器人学
规模类别：
- 10K<n<100K

# LeIsaac PickOrange — 预适配GR00T的预处理数据集

本数据集为[LightwheelAI/leisaac-pick-orange](https://huggingface.co/datasets/LightwheelAI/leisaac-pick-orange)的预处理版本，已适配GR00T N1.6微调任务。

## 与原始数据集的差异
原始数据集需经过多项修改才能适配GR00T N1.6，本版本已预先完成所有修复：

| 修复项 | 原始数据集 | 本数据集 |
|-----|----------|-------------|
| 视频编码格式 | AV1 | **H.264**（兼容decord） |
| 标注列 | 缺失 | 所有Parquet文件均已添加整型`task_index`（任务索引）列 |
| 模态配置文件（modality.json） | 缺失 | 已包含（包含状态、动作、视频、标注四类模态） |
| 格式 | LeRobot v2.1 | 兼容LeRobot v2.1格式 |

## 数据集详情
- **任务**：LeIsaac-SO101-PickOrange-v0（厨房场景，拾取3个橙子至餐盘）
- **机器人**：SO-101从动机械臂（5个臂关节+1个夹爪）
- **演示片段**：60次遥操作演示
- **总帧数**：约36,000
- **帧率**：30 FPS
- **相机配置**：前方相机（分辨率480×640）+ 腕部相机（分辨率480×640）
- **动作空间**：6维绝对关节角度（单位：度）

### 动作空间

索引 | 关节名称 | 角度范围（度）
------|----------------|-------------
0 | shoulder_pan（肩部平转关节） | [-38, 52]
1 | shoulder_lift（肩部抬升关节） | [-100, 64]
2 | elbow_flex（肘部屈伸关节） | [-99, 99]
3 | wrist_flex（腕部屈伸关节） | [22, 100]
4 | wrist_roll（腕部旋转关节） | [-14, 51]
5 | gripper（夹爪） | [1, 91]

### 模态配置文件（modality.json）
json
{
"state": {
"single_arm": {"start": 0, "end": 5},
"gripper": {"start": 5, "end": 6}
},
"action": {
"single_arm": {"start": 0, "end": 5},
"gripper": {"start": 5, "end": 6}
},
"video": {
"front": {"original_key": "observation.images.front"},
"wrist": {"original_key": "observation.images.wrist"}
},
"annotation": {
"human.task_description": {"original_key": "annotation.human.task_description"}
}
}

## GR00T使用方法
bash
# 1. 克隆GR00T仓库
git clone https://github.com/NVIDIA/Isaac-GR00T.git

# 2. 下载本数据集
huggingface-cli download rajeshramana/leisaac-pick-orange-prepared
--local-dir ./demo_data/pick_orange --repo-type dataset

# 3. 启动微调任务
python gr00t/experiment/launch_finetune.py
--base-model-path nvidia/GR00T-N1.6-3B
--dataset-path ./demo_data/pick_orange
--modality-config-path ./so101_pick_orange_config.py
--embodiment-tag NEW_EMBODIMENT
--num-gpus 1
--max-steps 10000
--no-tune-diffusion-model

## 已训练模型
本数据集经10,000步微调（损失值为0.017）的模型已上传至：[rajeshramana/groot-n1.6-pick-orange](https://huggingface.co/rajeshramana/groot-n1.6-pick-orange)

## 预处理步骤
1. **重编码所有120段视频**：将原AV1编码转换为H.264格式，使用命令：
bash
ffmpeg -y -i input.mp4 -c:v libx264 -crf 23 -preset fast -pix_fmt yuv420p output.mp4

2. **为全部60个演示片段的Parquet文件添加标注列**：
python
df["annotation.human.task_description"] = 0 # 整型任务索引task_index

3. **生成模态配置文件modality.json**：将扁平化的数据列映射至GR00T要求的模态结构。

## 数据集来源
- 原始数据集：[LightwheelAI/leisaac-pick-orange](https://huggingface.co/datasets/LightwheelAI/leisaac-pick-orange)
- 开发框架：[LightwheelAI/leisaac](https://github.com/LightwheelAI/leisaac)
- 仿真环境：NVIDIA Isaac Lab 2.3.2

## 引用格式
bibtex
@misc{leisaac-pick-orange-prepared,
title={LeIsaac PickOrange Prepared Dataset for GR00T},
author={Rajesh Kumar},
year={2026},
url={https://huggingface.co/datasets/rajeshramana/leisaac-pick-orange-prepared}
}