vlabench_primitive_ft_dataset

# Datacard
This is the official fine-tuning dataset provided by VLABench, with 500 episodes each task. The current version includes 10 primitive tasks.

## Source
- Project Page: [https://vlabench.github.io/](https://vlabench.github.io/)
- Arxiv Paper: [https://arxiv.org/abs/2412.18194](https://arxiv.org/abs/2412.18194)
- Code: [https://github.com/OpenMOSS/VLABench](https://github.com/OpenMOSS/VLABench)

## Uses
Download all archive files and use the following command to extract:
```sh
cat rdt_data.tar.gz.* | tar -xzvf -
```

In the resulting `VLABench_release` folder, there will be two folders: `primitive`(release now) and `composite`(under management).
In `primitive` folder, there are ten sub-folders and the dataset HDF5 files can be listed as:

```
VLABench_release
└── primitive
├── add_condiment
└── episode_0.hdf5
...
├── insert_flower
└── episode_0.hdf5
...
├── select_book
└── ...
├── select_chemistry_tube
└── ...
├── select_drink
└── ...
├── select_fruit
└── ...
├── select_mahjong
└── ...
├── select_painting
└── ...
├── select_poker
└── ...
└── select_toy
└── ...
```

An example of the single episode data is:
```
data
2025-02-23 20:46:40
instruction
(1,) |S38
['Please put the striped_10 in any hole.']
meta_info
entities
(6,) |S15
['billiards_table', 'striped_10', 'striped_14', 'striped_11', 'striped_12', 'solid_1']
episode_config
() | S1879
target_entity
(1,) |S10
['striped_10']
observation
depth
(212, 4, 480, 480) float32
ee_state
(212, 8) float32
point_cloud_colors
(212, 11905, 3) float32
point_cloud_points
(212, 11905, 3) float32
q_acceleration
(212, 7, 1) float32
q_state
(212, 7, 1) float32
q_velocity
(212, 7, 1) float32
rgb
(212, 4, 480, 480, 3) uint8
robot_mask
(212, 4, 480, 480) float32
trajectory
(212, 8) float32
```

## Citation
If you find our work helps,please cite us:
```
@misc{zhang2024vlabench,
title={VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks},
author={Shiduo Zhang and Zhe Xu and Peiju Liu and Xiaopeng Yu and Yuan Li and Qinghui Gao and Zhaoye Fei and Zhangyue Yin and Zuxuan Wu and Yu-Gang Jiang and Xipeng Qiu},
year={2024},
eprint={2412.18194},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2412.18194},
}
```

# 数据卡
本数据集为VLABench官方提供的微调数据集，每个任务包含500个回合。当前版本涵盖10项基础任务。

## 来源
- 项目页面：[https://vlabench.github.io/](https://vlabench.github.io/)
- arXiv论文：[https://arxiv.org/abs/2412.18194](https://arxiv.org/abs/2412.18194)
- 代码仓库：[https://github.com/OpenMOSS/VLABench](https://github.com/OpenMOSS/VLABench)

## 使用方式
下载所有归档文件后，使用如下命令进行解压：
sh
cat rdt_data.tar.gz.* | tar -xzvf -

解压后得到的`VLABench_release`文件夹中包含两个子文件夹：`primitive`（已发布）与`composite`（尚在管理中）。
在`primitive`文件夹下共有10个子目录，其数据集HDF5文件结构如下：

VLABench_release
└── primitive
├── add_condiment
└── episode_0.hdf5
...
├── insert_flower
└── episode_0.hdf5
...
├── select_book
└── ...
├── select_chemistry_tube
└── ...
├── select_drink
└── ...
├── select_fruit
└── ...
├── select_mahjong
└── ...
├── select_painting
└── ...
├── select_poker
└── ...
└── select_toy
└── ...

单个回合数据的示例结构如下：

data
2025-02-23 20:46:40
instruction
(1,) |S38
['请将striped_10放入任意球洞。']
meta_info
entities
(6,) |S15
['台球桌', 'striped_10', 'striped_14', 'striped_11', 'striped_12', 'solid_1']
episode_config
() | S1879
target_entity
(1,) |S10
['striped_10']
observation
depth
(212, 4, 480, 480) float32
ee_state
(212, 8) float32
point_cloud_colors
(212, 11905, 3) float32
point_cloud_points
(212, 11905, 3) float32
q_acceleration
(212, 7, 1) float32
q_state
(212, 7, 1) float32
q_velocity
(212, 7, 1) float32
rgb
(212, 4, 480, 480, 3) uint8
robot_mask
(212, 4, 480, 480) float32
trajectory
(212, 8) float32

## 引用方式
若您的工作用到本数据集，请引用如下文献：

@misc{zhang2024vlabench,
title={VLABench: 面向语言条件型机器人操作的大规模基准测试集，支持长时序推理任务},
author={张夺铎、徐哲、刘佩菊、于晓鹏、李源、高庆辉、费兆晔、尹张悦、吴祖轩、姜宇刚、邱锡鹏},
year={2024},
eprint={2412.18194},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2412.18194},
}