toto

This dataset was created using [LeRobot](https://github.com/huggingface/lerobot).

## Dataset Description

- **Homepage:** https://toto-benchmark.org/
- **Paper:** https://arxiv.org/abs/2306.00942
- **License:** mit

## Dataset Structure

[meta/info.json](meta/info.json):
```json
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 1003,
"total_frames": 325699,
"total_tasks": 1,
"total_videos": 1003,
"total_chunks": 2,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:1003"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4",
"features": {
"observation.images.image": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channel"
],
"video_info": {
"video.fps": 30.0,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"has_audio": false
}
},
"language_instruction": {
"dtype": "string",
"shape": [
1
],
"names": null
},
"observation.state": {
"dtype": "float32",
"shape": [
8
],
"names": {
"motors": [
"motor_0",
"motor_1",
"motor_2",
"motor_3",
"motor_4",
"motor_5",
"motor_6",
"motor_7"
]
}
},
"action": {
"dtype": "float32",
"shape": [
7
],
"names": {
"motors": [
"motor_0",
"motor_1",
"motor_2",
"motor_3",
"motor_4",
"motor_5",
"motor_6"
]
}
},
"timestamp": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"episode_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"frame_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"next.reward": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"next.done": {
"dtype": "bool",
"shape": [
1
],
"names": null
},
"index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"task_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
}
}
}
```

## Citation

**BibTeX:**

```bibtex
@inproceedings{zhou2023train,
author={Zhou, Gaoyue and Dean, Victoria and Srirama, Mohan Kumar and Rajeswaran, Aravind and Pari, Jyothish and Hatch, Kyle and Jain, Aryan and Yu, Tianhe and Abbeel, Pieter and Pinto, Lerrel and Finn, Chelsea and Gupta, Abhinav},
booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
title={Train Offline, Test Online: A Real Robot Learning Benchmark},
year={2023},
}
```

本数据集基于[LeRobot](https://github.com/huggingface/lerobot)构建。

## 数据集说明

- **主页**：https://toto-benchmark.org/
- **论文**：https://arxiv.org/abs/2306.00942
- **许可证**：MIT

## 数据集结构

[meta/info.json](meta/info.json):
json
{
"代码库版本": "v2.0",
"机器人类型": "未知",
"总回合数": 1003,
"总帧数": 325699,
"总任务数": 1,
"总视频数": 1003,
"总分块数": 2,
"单分块容量": 1000,
"帧率": 30,
"数据集划分": {
"训练集": "0:1003"
},
"数据存储路径": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"视频存储路径": "videos/chunk-{episode_chunk:03d}/{video_key}/episode_{episode_index:06d}.mp4",
"特征项": {
"观测.图像.图像": {
"数据类型": "视频",
"形状": [480, 640, 3],
"维度命名": ["高度", "宽度", "通道数"],
"视频信息": {
"视频帧率": 30.0,
"视频编码格式": "av1",
"视频像素格式": "yuv420p",
"是否为深度图": false,
"是否包含音频": false
}
},
"语言指令": {
"数据类型": "字符串",
"形状": [1],
"维度命名": null
},
"观测状态": {
"数据类型": "32位浮点型",
"形状": [8],
"维度命名": {
"电机": ["motor_0", "motor_1", "motor_2", "motor_3", "motor_4", "motor_5", "motor_6", "motor_7"]
}
},
"动作": {
"数据类型": "32位浮点型",
"形状": [7],
"维度命名": {
"电机": ["motor_0", "motor_1", "motor_2", "motor_3", "motor_4", "motor_5", "motor_6"]
}
},
"时间戳": {
"数据类型": "32位浮点型",
"形状": [1],
"维度命名": null
},
"回合索引": {
"数据类型": "64位整型",
"形状": [1],
"维度命名": null
},
"帧索引": {
"数据类型": "64位整型",
"形状": [1],
"维度命名": null
},
"下一时刻奖励": {
"数据类型": "32位浮点型",
"形状": [1],
"维度命名": null
},
"下一时刻终止标志": {
"数据类型": "布尔型",
"形状": [1],
"维度命名": null
},
"全局索引": {
"数据类型": "64位整型",
"形状": [1],
"维度命名": null
},
"任务索引": {
"数据类型": "64位整型",
"形状": [1],
"维度命名": null
}
}
}

## 引用

**BibTeX格式引用：**
bibtex
@inproceedings{zhou2023train,
author={Zhou, Gaoyue and Dean, Victoria and Srirama, Mohan Kumar and Rajeswaran, Aravind and Pari, Jyothish and Hatch, Kyle and Jain, Aryan and Yu, Tianhe and Abbeel, Pieter and Pinto, Lerrel and Finn, Chelsea and Gupta, Abhinav},
booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
title={Train Offline, Test Online: A Real Robot Learning Benchmark},
year={2023},
}

搜集汇总

背景与挑战

背景概述

toto数据集是基于TOTO基准构建的机器人学习资源，包含1003个视频片段和约5.64GB数据，适用于离线训练与在线测试场景。它采用MIT许可证，由LeRobot开发，提供了多维度观测和动作特征。