droid_100

This dataset was created using [LeRobot](https://github.com/huggingface/lerobot).

## Dataset Description

- **Homepage:** https://droid-dataset.github.io/
- **Paper:** https://arxiv.org/abs/2403.12945
- **License:** mit

## Dataset Structure

[meta/info.json](meta/info.json):
```json
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 100,
"total_frames": 32212,
"total_tasks": 47,
"total_videos": 300,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:100"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4",
"features": {
"observation.images.exterior_image_1_left": {
"dtype": "video",
"shape": [
180,
320,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 15.0,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
}
},
"observation.images.exterior_image_2_left": {
"dtype": "video",
"shape": [
180,
320,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 15.0,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
}
},
"observation.images.wrist_image_left": {
"dtype": "video",
"shape": [
180,
320,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 15.0,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
}
},
"language_instruction": {
"dtype": "string",
"shape": [
1
],
"names": null
},
"observation.state": {
"dtype": "float32",
"shape": [
7
],
"names": {
"motors": [
"motor_0",
"motor_1",
"motor_2",
"motor_3",
"motor_4",
"motor_5",
"motor_6"
]
}
},
"action": {
"dtype": "float32",
"shape": [
7
],
"names": {
"motors": [
"motor_0",
"motor_1",
"motor_2",
"motor_3",
"motor_4",
"motor_5",
"motor_6"
]
}
},
"timestamp": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"episode_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"frame_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"next.reward": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"next.done": {
"dtype": "bool",
"shape": [
1
],
"names": null
},
"index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"task_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
}
}
}
```

## Citation

**BibTeX:**

```bibtex
@article{khazatsky2024droid,
title = {DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset},
author = {Alexander Khazatsky and Karl Pertsch and Suraj Nair and Ashwin Balakrishna and Sudeep Dasari and Siddharth Karamcheti and Soroush Nasiriany and Mohan Kumar Srirama and Lawrence Yunliang Chen and Kirsty Ellis and Peter David Fagan and Joey Hejna and Masha Itkina and Marion Lepert and Yecheng Jason Ma and Patrick Tree Miller and Jimmy Wu and Suneel Belkhale and Shivin Dass and Huy Ha and Arhan Jain and Abraham Lee and Youngwoon Lee and Marius Memmel and Sungjae Park and Ilija Radosavovic and Kaiyuan Wang and Albert Zhan and Kevin Black and Cheng Chi and Kyle Beltran Hatch and Shan Lin and Jingpei Lu and Jean Mercat and Abdul Rehman and Pannag R Sanketi and Archit Sharma and Cody Simpson and Quan Vuong and Homer Rich Walke and Blake Wulfe and Ted Xiao and Jonathan Heewon Yang and Arefeh Yavary and Tony Z. Zhao and Christopher Agia and Rohan Baijal and Mateo Guaman Castro and Daphne Chen and Qiuyu Chen and Trinity Chung and Jaimyn Drake and Ethan Paul Foster and Jensen Gao and David Antonio Herrera and Minho Heo and Kyle Hsu and Jiaheng Hu and Donovon Jackson and Charlotte Le and Yunshuang Li and Kevin Lin and Roy Lin and Zehan Ma and Abhiram Maddukuri and Suvir Mirchandani and Daniel Morton and Tony Nguyen and Abigail O'Neill and Rosario Scalise and Derick Seale and Victor Son and Stephen Tian and Emi Tran and Andrew E. Wang and Yilin Wu and Annie Xie and Jingyun Yang and Patrick Yin and Yunchu Zhang and Osbert Bastani and Glen Berseth and Jeannette Bohg and Ken Goldberg and Abhinav Gupta and Abhishek Gupta and Dinesh Jayaraman and Joseph J Lim and Jitendra Malik and Roberto Martín-Martín and Subramanian Ramamoorthy and Dorsa Sadigh and Shuran Song and Jiajun Wu and Michael C. Yip and Yuke Zhu and Thomas Kollar and Sergey Levine and Chelsea Finn},
year = {2024},
}
```

本数据集基于[LeRobot](https://github.com/huggingface/lerobot)构建。

## 数据集说明
- **主页**：https://droid-dataset.github.io/
- **论文**：https://arxiv.org/abs/2403.12945
- **许可证**：MIT

## 数据集结构
`[meta/info.json](meta/info.json)`：
json
{
"代码库版本": "v2.0",
"机器人类型": "未知",
"总轨迹数": 100,
"总帧数": 32212,
"总任务数": 47,
"总视频数": 300,
"总数据块数": 1,
"单数据块大小": 1000,
"帧率": 15,
"数据集划分": {
"训练集": "0:100"
},
"数据文件路径格式": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"视频文件路径格式": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4",
"数据特征": {
"观测图像.左侧外景相机1": {
"数据类型": "video",
"形状": [180, 320, 3],
"维度命名": ["高度", "宽度", "通道数"],
"视频信息": {
"视频帧率": 15.0,
"视频编码格式": "av1",
"视频像素格式": "yuv420p",
"是否为深度图": false,
"是否含音频": false
}
},
"观测图像.左侧外景相机2": {
"数据类型": "video",
"形状": [180, 320, 3],
"维度命名": ["高度", "宽度", "通道数"],
"视频信息": {
"视频帧率": 15.0,
"视频编码格式": "av1",
"视频像素格式": "yuv420p",
"是否为深度图": false,
"是否含音频": false
}
},
"观测图像.左侧腕部相机": {
"数据类型": "video",
"形状": [180, 320, 3],
"维度命名": ["高度", "宽度", "通道数"],
"视频信息": {
"视频帧率": 15.0,
"视频编码格式": "av1",
"视频像素格式": "yuv420p",
"是否为深度图": false,
"是否含音频": false
}
},
"语言指令": {
"数据类型": "string",
"形状": [1],
"维度命名": null
},
"观测状态": {
"数据类型": "float32",
"形状": [7],
"维度命名": {
"电机": ["motor_0", "motor_1", "motor_2", "motor_3", "motor_4", "motor_5", "motor_6"]
}
},
"动作指令": {
"数据类型": "float32",
"形状": [7],
"维度命名": {
"电机": ["motor_0", "motor_1", "motor_2", "motor_3", "motor_4", "motor_5", "motor_6"]
}
},
"时间戳": {
"数据类型": "float32",
"形状": [1],
"维度命名": null
},
"轨迹索引": {
"数据类型": "int64",
"形状": [1],
"维度命名": null
},
"帧索引": {
"数据类型": "int64",
"形状": [1],
"维度命名": null
},
"下一时刻奖励": {
"数据类型": "float32",
"形状": [1],
"维度命名": null
},
"下一时刻终止标志": {
"数据类型": "bool",
"形状": [1],
"维度命名": null
},
"全局索引": {
"数据类型": "int64",
"形状": [1],
"维度命名": null
},
"任务索引": {
"数据类型": "int64",
"形状": [1],
"维度命名": null
}
}
}

## 引用
**BibTeX格式**：
bibtex
@article{khazatsky2024droid,
title = {DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset},
author = {Alexander Khazatsky and Karl Pertsch and Suraj Nair and Ashwin Balakrishna and Sudeep Dasari and Siddharth Karamcheti and Soroush Nasiriany and Mohan Kumar Srirama and Lawrence Yunliang Chen and Kirsty Ellis and Peter David Fagan and Joey Hejna and Masha Itkina and Marion Lepert and Yecheng Jason Ma and Patrick Tree Miller and Jimmy Wu and Suneel Belkhale and Shivin Dass and Huy Ha and Arhan Jain and Abraham Lee and Youngwoon Lee and Marius Memmel and Sungjae Park and Ilija Radosavovic and Kaiyuan Wang and Albert Zhan and Kevin Black and Cheng Chi and Kyle Beltran Hatch and Shan Lin and Jingpei Lu and Jean Mercat and Abdul Rehman and Pannag R Sanketi and Archit Sharma and Cody Simpson and Quan Vuong and Homer Rich Walke and Blake Wulfe and Ted Xiao and Jonathan Heewon Yang and Arefeh Yavary and Tony Z. Zhao and Christopher Agia and Rohan Baijal and Mateo Guaman Castro and Daphne Chen and Qiuyu Chen and Trinity Chung and Jaimyn Drake and Ethan Paul Foster and Jensen Gao and David Antonio Herrera and Minho Heo and Kyle Hsu and Jiaheng Hu and Donovon Jackson and Charlotte Le and Yunshuang Li and Kevin Lin and Roy Lin and Zehan Ma and Abhiram Maddukuri and Suvir Mirchandani and Daniel Morton and Tony Nguyen and Abigail O'Neill and Rosario Scalise and Derick Seale and Victor Son and Stephen Tian and Emi Tran and Andrew E. Wang and Yilin Wu and Annie Xie and Jingyun Yang and Patrick Yin and Yunchu Zhang and Osbert Bastani and Glen Berseth and Jeannette Bohg and Ken Goldberg and Abhinav Gupta and Abhishek Gupta and Dinesh Jayaraman and Joseph J Lim and Jitendra Malik and Roberto Martín-Martín and Subramanian Ramamoorthy and Dorsa Sadigh and Shuran Song and Jiajun Wu and Michael C. Yip and Yuke Zhu and Thomas Kollar and Sergey Levine and Chelsea Finn},
year = {2024},
}

搜集汇总

背景与挑战

背景概述

droid_100是一个用于机器人操作研究的大规模数据集，包含100个episodes、47个任务和300个视频，视频帧率为15fps。数据集结构丰富，涵盖多种观测数据和元数据，适用于机器人学习和控制任务。