heesunny/ur5e_koras_black_block_pick

# UR5e Black Block Pick and Place — KORAS AI TOOLS **Author**: Heesun Lee (KORASROBOTICS) **Date**: 2026-04-17 **Purpose**: Isaac Sim Lab Mimic 데이터 증강 + Pi0 VLA 학습용 ## Task - **Task Name**: `black_block_pick` - **Task Instruction**: `pick up the black block and place in basket` - **Episodes**: 30 - **Record Hz**: 10 ## Robot Setup - **Robot**: UR5e (192.168.3.12) - **Gripper**: KORAS KR-DATC 평행 2지 그리퍼 (Modbus RTU, 0-999) - Z축 기구적 compliance 없음 (순응형 아님) - **Teleoperation**: GELLO 7DOF (Dynamixel x7) - **Global Camera**: RealSense D435i (SN: 346522071588) - **Wrist Camera**: RealSense D405 (SN: 335122270150) - **FTS**: UR5e 내장 (RTDE getActualTCPForce, 이동 중 노이즈 ±19.5N) ## Data Schema (per episode) ### Top-level (data.json) | Field | Type | Description | |-------|------|-------------| | episode_id | int | 0-based sequential | | task | str | Language instruction | | task_name | str | Task identifier | | termination_reason | str | "user_stop" | | num_steps | int | Step count | | num_frames | int | Image count (== num_steps) | | record_hz | int | 10 Hz | | skip_counts | dict | {missing_camera, stale_frame, nan_state} | | camera_config | dict | Camera serial, model, resolution, fps | | steps | list | Per-step data | ### Per-step | Field | Shape | Description | |-------|-------|-------------| | state | [7] | 6 joint angles (rad) + 1 gripper (0=closed, 1=open) | | action | [7] | GELLO commanded joints + gripper | | force_torque | [6] | Fx, Fy, Fz, Tx, Ty, Tz (N, N·m) | | tcp_pose | [6] | x, y, z, rx, ry, rz (m, rad) | | camera_timestamps | dict | {global, wrist} epoch seconds | | camera_frame_ids | dict | {global, wrist} RealSense frame counter | | camera_state_time_gap | dict | Camera-state timestamp difference | | timestamp | float | Epoch seconds | | step_index | int | Sequential 0-based | ### Images - `images/global/XXXXX.jpg` — 224x224 JPEG (D435i overhead view) - `images/wrist/XXXXX.jpg` — 224x224 JPEG (D405 wrist-mounted view) ## Camera Config (saved in data.json) ```json { "global": {"serial": "346522071588", "model": "D435i", "raw_resolution": [640, 480], "fps": 30, "save_resolution": [224, 224]}, "wrist": {"serial": "335122270150", "model": "D405", "raw_resolution": [640, 480], "fps": 30, "save_resolution": [224, 224]} } ``` ## Camera Intrinsics (from isaac_sim_integration_spec.md, 5x measured) - **D435i**: fx=604.71, fy=604.54, cx=325.06, cy=247.84 - **D405**: fx=392.74, fy=392.52, cx=323.59, cy=238.37 ## Data Quality (30 episodes) | Metric | Min | Max | Mean | |--------|-----|-----|------| | Steps | 93 | 117 | 101 | | Duration | 11.7s | 14.9s | 12.8s | | Action jump max | 0.112 | 0.298 | 0.19 | | Gripper closed % | 38.4% | 45.8% | 42.0% | | FTS norm mean | 15.2N | 20.1N | 17.3N | ## Position Variation (for Isaac Sim randomization) | | Min | Max | Range | |---|-----|-----|-------| | Grasp X | 259mm | 681mm | **422mm** | | Grasp Y | -246mm | 76mm | **322mm** | | Grasp Z | 175mm | 193mm | 18mm | | Place X | 334mm | 616mm | **281mm** | | Place Y | -198mm | 29mm | **227mm** | ## Isaac Sim Integration Notes - 실측 스펙 문서: `analysis_reports/isaac_sim_integration_spec.md` (GitHub: heesunny99/koras-ai-tools) - UR5e joint limits, camera intrinsics, FTS noise 전부 5회 실측 기반 - 카메라 extrinsics (로봇 base 기준 마운트 위치)는 별도 캘리브레이션 필요 - UR5e URDF: 공개 (Universal Robots GitHub) - 그리퍼 CAD: KORAS 내부 제공 ## Training Pipeline ```bash cd ~/openpi # 1. Convert to LeRobot format .venv/bin/python scripts/convert_bc_data.py --repo-id koras/ur5e_black_block_pick # 2. Compute normalization stats .venv/bin/python scripts/compute_norm_stats.py --config-name pi0_ur5e_lora # 3. Train Pi0 LoRA .venv/bin/python scripts/train.py pi0_ur5e_lora --exp-name black_block_pick_v1 ``` ## License Apache 2.0 — KORASROBOTICS CO., LTD.