vlabench_primitive_ft_lerobot_video

# VLABench Primitive Tasks Dataset - LeRobot v3.0 ## Dataset Description This dataset is organized in the **LeRobot v3.0** format and is used for integrating **VLABench** into the **LeRobot** framework officially. Compared with the **v2.0** version and the **RLDS** version of the dataset, this release stores visual observations in a **video-compressed format** rather than as individual image files. This design provides significant advantages in both **storage efficiency** and **data loading performance**. ## Task Coverage This dataset includes 10 primitive tasks: - `add_condiment` - `insert_flower` - `select_book` - `select_chemistry_tube` - `select_drink` - `select_fruit` - `select_mahjong` - `select_painting` - `select_poker` - `select_toy` Models trained on this dataset can be further evaluated on multiple generalization tracks in VLABench. Baseline models are provided at: - [VLABench/pi05-primitive-10task](https://huggingface.co/VLABench/pi05-primitive-10task) - [VLABench/pi0-fast-primitive-10task](https://huggingface.co/VLABench/pi0-fast-primitive-10task) - [VLABench/pi0-primitive-10task](https://huggingface.co/VLABench/pi0-primitive-10task) ## Dataset Structure The dataset structure follows the metadata specification in [`meta/info.json`](https://huggingface.co/datasets/VLABench/vlabench_primitive_ft_lerobot_video/blob/main/meta/info.json). ```json { "codebase_version": "v3.0", "robot_type": "panda", "total_episodes": 5000, "total_frames": 575101, "total_tasks": 128, "chunks_size": 1000, "data_files_size_in_mb": 100, "video_files_size_in_mb": 500, "fps": 10, "splits": { "train": "0:5000" }, "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet", "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4", "features": { "image": { "dtype": "video", "shape": [480, 480, 3], "names": ["height", "width", "channel"], "fps": 10, "info": { "video.height": 480, "video.width": 480, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 10, "video.channels": 3, "has_audio": false } }, "second_image": { "dtype": "video", "shape": [480, 480, 3], "names": ["height", "width", "channel"], "fps": 10, "info": { "video.height": 480, "video.width": 480, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 10, "video.channels": 3, "has_audio": false } }, "wrist_image": { "dtype": "video", "shape": [480, 480, 3], "names": ["height", "width", "channel"], "fps": 10, "info": { "video.height": 480, "video.width": 480, "video.codec": "av1", "video.pix_fmt": "yuv420p", "video.is_depth_map": false, "video.fps": 10, "video.channels": 3, "has_audio": false } }, "state": { "dtype": "float32", "shape": [7], "names": ["state"], "fps": 10 }, "actions": { "dtype": "float32", "shape": [7], "names": ["actions"], "fps": 10 }, "timestamp": { "dtype": "float32", "shape": [1], "names": null }, "frame_index": { "dtype": "int64", "shape": [1], "names": null }, "episode_index": { "dtype": "int64", "shape": [1], "names": null }, "index": { "dtype": "int64", "shape": [1], "names": null }, "task_index": { "dtype": "int64", "shape": [1], "names": null } } } ``` ## Citation If you use this dataset, please cite: ```bibtex @article{zhang2024vlabench, title={Vlabench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks}, author={Zhang, Shiduo and Xu, Zhe and Liu, Peiju and Yu, Xiaopeng and Li, Yuan and Gao, Qinghui and Fei, Zhaoye and Yin, Zhangyue and Wu, Zuxuan and Jiang, Yu-Gang and others}, journal={arXiv preprint arXiv:2412.18194}, year={2024} }