phy_pick_and_place_cup

# phy_pick_and_place_cup Paired simulation and real-world UR5 cup pick-and-place videos for sim-to-real video reconstruction. ## Contents - 140 quality-checked episodes; train 118, validation 10, test 12. - Each episode contains `sim.mp4`, `real.mp4`, and `first_frame.png`. - Both videos are H.264 RGB, 640 x 480, 5 fps, with identical original frame counts (90-164). - `first_frame.png` is the original decoded first real-world frame, before lossy H.264 re-encoding. - `manifest.jsonl` records per-episode paths, timestamps, frame counts, SHA-256 checksums, and byte sizes. - `split.json` preserves the frozen episode-level split (seed 0). - `exclusions.json` records rejected episodes; rejected media are not included. ## Download And Extract Download `phy_pick_and_place_cup.zip` from this repository and extract it: ```bash unzip phy_pick_and_place_cup.zip -d phy_pick_and_place_cup cd phy_pick_and_place_cup sha256sum -c SHA256SUMS.txt ``` ```text episodes/episode_000004/ real.mp4 sim.mp4 first_frame.png manifest.jsonl split.json dataset_info.json exclusions.json SHA256SUMS.txt ``` ## Data Preparation Real video source: the Berkeley Autolab UR5 dataset (`berkeley_autolab_ur5`). Simulation videos replay saved episode states without new physics integration. Real and simulation frames are aligned using recorded timestamps and the full-precision capture-state offset. All exported files were checked against their construction hashes; both video streams were fully decoded to verify frame counts. Eleven episodes failed contact checks and one additional episode failed visual review. The archive includes only the final accepted episodes, without duplicate first-81-frame training clips or segmentation labels. ## Intended Use Input: simulation RGB video and the first real-world frame. Target: the same episode's complete real-world video. The paired reconstruction task does not establish independent action-prediction ability. Preserve the episode-level split when deriving training windows. Original dataset terms continue to apply to the real-world source data. ## Wrist Camera View / 手腕视角 The same **140 episodes** are also available from the wrist camera: train 118 / val 10 / test 12, with identical episode names and frame counts. Download **`wrist/phy_pick_and_place_cup_wrist.zip`** and extract it into a separate directory. Each episode contains wrist `real.mp4`, `sim.mp4`, and `first_frame.png` at 640 x 480, 5 fps. See `wrist/README.md` for usage and `wrist/preview_episode_000130.jpg` for a real/simulation preview. Wrist metadata and checksums are under `wrist/`; the original root-level archive and metadata describe the external view. **Calibration limitation:** the wrist camera uses the existing approximate gripper-based calibration, which has not been independently verified. Timing assumes the same fitted capture-state offset as the external stream. Precise wrist alignment is not guaranteed. 新增手腕视角包沿用同一批数据及划分；手腕标定为已有近似结果，共享时间偏移未经独立验证，具体限制见 `wrist/README.md`。