Aren99/G1_Dex1_DiverseManip_DualArm_128x128

--- license: apache-2.0 task_categories: - robotics tags: - LeRobot configs: - config_name: default data_files: data/*/*.parquet --- This dataset was created using [LeRobot](https://github.com/huggingface/lerobot). ## Dataset Description - **Homepage:** [unitreerobotics](https://github.com/unitreerobotics) - **License:** apache-2.0 - **Task Objective:** Organize and tidy the items on the table. - **Operation Duration:** Each operation takes approximately 20 to 40 seconds. - **Recording Frequency:** 30 Hz. - **Robot Type:** 7-DOF dual-arm G1 robot. - **End Effector:** Gripper. - **Dual-Arm Operation:** Yes. - **Image Resolution:** 128x128. - **Camera Positions:** head-mounted (binocular cameras). - **Data Content:** • Robot's current state. • Robot's next action. • Current camera view images. - **Robot Initial Posture:** The first robot state in each dataset entry. - **Object Placement:** Randomly placed within the robot arm's motion range and the field of view of the robot's head-mounted camera. - **Camera View:** Follow the guidelines in **Part 5** of [AVP Teleoperation Documentation](https://github.com/unitreerobotics/avp_teleoperate). <table> <tr> <td><img src="assets/4.gif" width="200px" /></td> <td><img src="assets/1.gif" width="200px" /></td> <td><img src="assets/2.gif" width="200px" /></td> <td><img src="assets/3.gif" width="200px" /></td> </tr> </table> - **Important Notes:** 1. This is a G1 diversity dataset that can be used for video generation models, world models, and other applications \[[Lee et al., 2018](#citation)\]. 2. If you want to use the lerobotv2.1 format, refer to this file for conversion: [convert_v3_to_v2.py](https://github.com/NVIDIA/Isaac-GR00T/blob/main/scripts/lerobot_conversion/convert_v3_to_v2.py) 3. Due to the inability to precisely describe spatial positions, adjust the scene to closely match the first frame of the dataset after installing the hardware as specified in **Part 5** of [AVP Teleoperation Documentation](https://github.com/unitreerobotics/avp_teleoperate). 4. Data collection is not completed in a single session, and variations between data entries exist. Ensure these variations are accounted for during model training. ## Dataset Structure [meta/info.json](meta/info.json): ```json { "codebase_version": "v3.0", "robot_type": "Unitree_G1_Dex1", "total_episodes": 525, "total_frames": 413538, "total_tasks": 1, "chunks_size": 1000, "data_files_size_in_mb": 100, "video_files_size_in_mb": 500, "fps": 30, "splits": { "train": "0:525" }, "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet", "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4", "features": { "observation.state": { "dtype": "float32", "shape": [ 16 ], "names": [ [ "kLeftShoulderPitch", "kLeftShoulderRoll", "kLeftShoulderYaw", "kLeftElbow", "kLeftWristRoll", "kLeftWristPitch", "kLeftWristYaw", "kRightShoulderPitch", "kRightShoulderRoll", "kRightShoulderYaw", "kRightElbow", "kRightWristRoll", "kRightWristPitch", "kRightWristYaw", "kLeftGripper", "kRightGripper" ] ] }, "action": { "dtype": "float32", "shape": [ 16 ], "names": [ [ "kLeftShoulderPitch", "kLeftShoulderRoll", "kLeftShoulderYaw", "kLeftElbow", "kLeftWristRoll", "kLeftWristPitch", "kLeftWristYaw", "kRightShoulderPitch", "kRightShoulderRoll", "kRightShoulderYaw", "kRightElbow", "kRightWristRoll", "kRightWristPitch", "kRightWristYaw", "kLeftGripper", "kRightGripper" ] ] }, "observation.images.cam_left_high": { "dtype": "video", "shape": [ 128, 128, 3 ], "names": [ "height", "width", "channel" ], "info": { "video.height": 128, "video.width": 128, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false } }, "observation.images.cam_right_high": { "dtype": "video", "shape": [ 128, 128, 3 ], "names": [ "height", "width", "channel" ], "info": { "video.height": 128, "video.width": 128, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false } }, "timestamp": { "dtype": "float32", "shape": [ 1 ], "names": null }, "frame_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "episode_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "task_index": { "dtype": "int64", "shape": [ 1 ], "names": null } } } ``` ## Citation **BibTeX:** ```bibtex @article{lee2018stochastic, title={Stochastic Adversarial Video Prediction}, author={Lee, Alex X. and Zhang, Richard and Ebert, Frederik and Abbeel, Pieter and Finn, Chelsea and Levine, Sergey}, journal={arXiv preprint arXiv:1804.01523}, year={2018}, url={https://arxiv.org/abs/1804.01523} } ```--- license: apache-2.0 任务类别： - 机器人学（robotics） 标签： - LeRobot（LeRobot） 配置项： - 配置名称：default 数据文件：data/*/*.parquet --- 本数据集基于[LeRobot（LeRobot）](https://github.com/huggingface/lerobot)构建。 ## 数据集描述 - **主页**：[unitreerobotics](https://github.com/unitreerobotics) - **许可证**：apache-2.0 - **任务目标**：整理桌面物品。 - **单次操作时长**：每轮操作耗时约20至40秒。 - **录制帧率**：30Hz。 - **机器人型号**：7自由度（7-DOF）双臂G1机器人。 - **末端执行器**：夹爪。 - **双臂操作**：支持。 - **图像分辨率**：128×128。 - **相机安装位置**：头部搭载双目相机。 - **数据内容**： • 机器人当前状态 • 机器人下一动作指令 • 当前相机视角的图像数据 - **机器人初始姿态**：每个数据集条目的首个机器人状态。 - **物体摆放规则**：物品随机放置于机械臂运动范围内且处于头部相机的视场之中。 - **相机视角规范**：遵循[AVP远程操作文档（AVP Teleoperation Documentation）](https://github.com/unitreerobotics/avp_teleoperate)第5部分的指导方针。 <table> <tr> <td><img src="assets/4.gif" width="200px" /></td> <td><img src="assets/1.gif" width="200px" /></td> <td><img src="assets/2.gif" width="200px" /></td> <td><img src="assets/3.gif" width="200px" /></td> </tr> </table> - **重要注意事项**： 1. 本数据集为G1多样性数据集，可用于视频生成模型、世界模型等相关应用[Lee等, 2018](#citation)。 2. 若需使用LeRobot v2.1格式，请参考该转换脚本：[convert_v3_to_v2.py](https://github.com/NVIDIA/Isaac-GR00T/blob/main/scripts/lerobot_conversion/convert_v3_to_v2.py) 3. 由于无法精准描述空间位置，请在按照[AVP远程操作文档（AVP Teleoperation Documentation）](https://github.com/unitreerobotics/avp_teleoperate)第5部分的要求完成硬件安装后，将实验场景调整至与数据集首帧高度匹配。 4. 本次数据采集并非单次会话完成，不同数据条目间存在差异，在模型训练阶段需充分考虑此类差异。 ## 数据集结构 [meta/info.json]： json { "codebase_version": "v3.0", "robot_type": "Unitree_G1_Dex1", "total_episodes": 525, "total_frames": 413538, "total_tasks": 1, "chunks_size": 1000, "data_files_size_in_mb": 100, "video_files_size_in_mb": 500, "fps": 30, "splits": { "train": "0:525" }, "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet", "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4", "features": { "observation.state": { "dtype": "float32", "shape": [ 16 ], "names": [ [ "kLeftShoulderPitch", "kLeftShoulderRoll", "kLeftShoulderYaw", "kLeftElbow", "kLeftWristRoll", "kLeftWristPitch", "kLeftWristYaw", "kRightShoulderPitch", "kRightShoulderRoll", "kRightShoulderYaw", "kRightElbow", "kRightWristRoll", "kRightWristPitch", "kRightWristYaw", "kLeftGripper", "kRightGripper" ] ] }, "action": { "dtype": "float32", "shape": [ 16 ], "names": [ [ "kLeftShoulderPitch", "kLeftShoulderRoll", "kLeftShoulderYaw", "kLeftElbow", "kLeftWristRoll", "kLeftWristPitch", "kLeftWristYaw", "kRightShoulderPitch", "kRightShoulderRoll", "kRightShoulderYaw", "kRightElbow", "kRightWristRoll", "kRightWristPitch", "kRightWristYaw", "kLeftGripper", "kRightGripper" ] ] }, "observation.images.cam_left_high": { "dtype": "video", "shape": [ 128, 128, 3 ], "names": [ "height", "width", "channel" ], "info": { "video.height": 128, "video.width": 128, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false } }, "observation.images.cam_right_high": { "dtype": "video", "shape": [ 128, 128, 3 ], "names": [ "height", "width", "channel" ], "info": { "video.height": 128, "video.width": 128, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 30, "video.channels": 3, "has_audio": false } }, "timestamp": { "dtype": "float32", "shape": [ 1 ], "names": null }, "frame_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "episode_index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "index": { "dtype": "int64", "shape": [ 1 ], "names": null }, "task_index": { "dtype": "int64", "shape": [ 1 ], "names": null } } } ## 参考文献 **BibTeX（BibTeX）格式：** bibtex @article{lee2018stochastic, title={随机对抗视频预测}, author={Lee, Alex X. and Zhang, Richard and Ebert, Frederik and Abbeel, Pieter and Finn, Chelsea and Levine, Sergey}, journal={arXiv预印本 arXiv:1804.01523}, year={2018}, url={https://arxiv.org/abs/1804.01523} }