MicroAGI00

---

# MicroAGI00: MicroAGI Egocentric Dataset (2025)

> License: MicroAGI00 Open Use, No-Resale v1.0 (see `LICENSE`).
> No resale: You may not sell or paywall this dataset or derivative data. Trained models/outputs may be released under any terms.

## Overview
MicroAGI00 is a large-scale egocentric RGB+D dataset of human manipulation in https://behavior.stanford.edu/challenge/index.html tasks.

## Quick facts

* Modality: synchronized RGB + 16‑bit depth + IMU + annotations
* Resolution & rate (RGB): 1920×1080 @ 30 FPS (in MCAP)
* Depth: 16‑bit, losslessly compressed inside MCAP
* Scale: ≈1,000,000 synchronized RGB frames and ≈1,000,000 depth frames (≈1M frame pairs)
* Container: `.mcap` (all signals + annotations)
* Previews: For as sample for only some bags `.mp4` per sequence (annotated RGB; visualized native depth)
* Annotations: Only in %5 of the dataset, hand landmarks and short action text

## What’s included per sequence

* One large **MCAP** file containing:

* RGB frames (1080p/30 fps)
* 16‑bit depth stream (lossless compression)
* IMU data (as available)
* For Some Data the Embedded annotations (hands, action text)

**MP4** preview videos:

* Annotated RGB (for quick review)
* Visualized native depth map (for quick review)

> Note: MP4 previews may be lower quality than MCAP due to compression and post‑processing. Research use should read from MCAP.

## Annotations

Annotations are generated by our in‑house.

### Hand annotations 21 Joints (not all shown below as it would be too long) (per frame) — JSON schema example

```
{
"frame_number": 9,
"timestamp_seconds": 0.3,
"resolution": { "width": 1920, "height": 1080 },
"hands": [
{
"hand_index": 0,
"landmarks": [
{ "id": 0, "name": "WRIST", "x": 0.7124036550521851, "y": 0.7347621917724609, "z": -1.444301744868426e-07, "visibility": 0.0 },

],
"hand": "Left",
"confidence": 0.9268525838851929
},
{
"hand_index": 1,
"landmarks": [
{ "id": 0, "name": "WRIST", "x": 0.4461262822151184, "y": 0.35183972120285034, "z": -1.2342320587777067e-07, "visibility": 0.0 },

"hand": "Right",
"confidence": 0.908446729183197
}
],
"frame_idx": 9,
"exact_frame_timestamp": 1758122341583104000,
"exact_frame_timestamp_sec": 1758122341.583104
}
```

### Text (action) annotations (per frame/window) — JSON schema example

```
{
"schema_version": "v1.0",
"action_text": "Right hand, holding a knife, is chopping cooked meat held by the left hand on the red cutting board.",
"confidence": 1.0,
"source": { "model": "MicroAGI, MAGI01" },
"exact_frame_timestamp": 1758122341583104000,
"exact_frame_timestamp_sec": 1758122341.583104
}
```

## Data access and structure

* Each top-level sample folder contains: One folder of strong heavy mcap dump, one folder of annotated mcap dump, one folder of mp4 previews
* All authoritative signals and annotations are inside the MCAP. Use the MP4s for quick visual QA only.

## Getting started

* Inspect an MCAP: `mcap info your_sequence.mcap`
* Extract messages: `mcap cat --topics <topic> your_sequence.mcap > out.bin`
* Python readers: `pip install mcap` (see the MCAP Python docs) or any MCAP-compatible tooling. Typical topics include RGB, depth, IMU, and annotation channels.

## Intended uses

* Policy and skill learning (robotics/VLA)
* Action detection and segmentation
* Hand/pose estimation and grasp analysis
* Depth-based reconstruction, SLAM, scene understanding
* World-model pre-post training

## Services and custom data

MicroAGI provides on-demand:

* Real‑to‑Sim pipelines
* ML‑enhanced 3D point clouds and SLAM reconstructions
* New data capture via our network of skilled tradespeople and factory workers (often below typical market cost)
* Enablement for your workforce to wear our device and run through our processing pipeline

Typical lead times: under two weeks (up to four weeks for large jobs).

## How to order more

Email `[e-mail]` with:

* Task description
* Desired hours or frame counts
* Proposed price
We will reply within one business day with lead time and final pricing.

Questions: `[e-mail]`

## License

This dataset is released under the MicroAGI00 Open Use, No‑Resale License v1.0 (custom). See [`LICENSE`](./LICENSE). Redistribution must be free‑of‑charge under the same license. Required credit: "This work uses the MicroAGI00 dataset (MicroAGI, 2025)."

## Attribution reminder

Public uses of the Dataset or Derivative Data must include the credit line above in a reasonable location for the medium (papers, repos, product docs, dataset pages, demo descriptions). Attribution is appreciated but not required for Trained Models or Outputs.

# MicroAGI00：MicroAGI 第一视角数据集（2025）

> 授权协议：MicroAGI00 非转售开放使用协议v1.0（详见`LICENSE`文件）。
> 非转售条款：您不得将本数据集或其衍生数据进行售卖或设置付费墙。基于本数据集训练得到的模型/输出可采用任意协议发布。

## 概述
MicroAGI00 是一个大规模人类操作第一视角RGB+D数据集，数据采集自[斯坦福行为挑战平台](https://behavior.stanford.edu/challenge/index.html)的任务场景。

## 核心参数
* 模态：同步RGB图像 + 16位深度图 + 惯性测量单元（Inertial Measurement Unit，IMU）数据 + 标注信息
* RGB分辨率与帧率：1920×1080，30帧/秒（存储格式为MCAP）
* 深度图：16位精度，在MCAP文件内采用无损压缩
* 数据规模：约100万对同步RGB与深度图帧（总计约100万RGB帧、100万深度图帧）
* 存储容器：`.mcap`格式（包含所有信号与标注信息）
* 预览文件：仅部分序列提供`.mp4`格式预览样本（含标注RGB帧与可视化原生深度图）
* 标注覆盖：仅5%的数据集包含手部关键点与简短动作文本标注

## 单序列包含内容
每个序列包含以下内容：
* 一个大型**MCAP**文件，内含：
* RGB帧（1080p分辨率，30帧/秒）
* 16位深度数据流（采用无损压缩）
* 惯性测量单元数据（视采集情况提供）
* 部分数据内置的标注信息（手部关键点、动作文本）

**MP4**格式预览视频：
* 带标注的RGB帧（用于快速预览）
* 原生深度图可视化结果（用于快速预览）

> 注意：由于压缩与后处理操作，MP4预览文件的画质可能低于MCAP原始文件。学术研究应优先读取MCAP格式数据。

## 标注说明
本数据集的标注均由团队自研流程生成。

### 手部标注（21个关节点，因篇幅限制未完全展示下方示例）——每帧标注的JSON schema示例

{
"frame_number": 9,
"timestamp_seconds": 0.3,
"resolution": { "width": 1920, "height": 1080 },
"hands": [
{
"hand_index": 0,
"landmarks": [
{ "id": 0, "name": "WRIST", "x": 0.7124036550521851, "y": 0.7347621917724609, "z": -1.444301744868426e-07, "visibility": 0.0 },

],
"hand": "Left",
"confidence": 0.9268525838851929
},
{
"hand_index": 1,
"landmarks": [
{ "id": 0, "name": "WRIST", "x": 0.4461262822151184, "y": 0.35183972120285034, "z": -1.2342320587777067e-07, "visibility": 0.0 },

"hand": "Right",
"confidence": 0.908446729183197
}
],
"frame_idx": 9,
"exact_frame_timestamp": 1758122341583104000,
"exact_frame_timestamp_sec": 1758122341.583104
}

### 文本（动作）标注（每帧/每窗口）——JSON schema示例

{
"schema_version": "v1.0",
"action_text": "Right hand, holding a knife, is chopping cooked meat held by the left hand on the red cutting board.",
"confidence": 1.0,
"source": { "model": "MicroAGI, MAGI01" },
"exact_frame_timestamp": 1758122341583104000,
"exact_frame_timestamp_sec": 1758122341.583104
}

## 数据访问与组织结构
* 每个顶级样本文件夹包含：原始MCAP数据文件夹、带标注MCAP数据文件夹、MP4预览文件文件夹
* 所有权威信号与标注信息均存储于MCAP文件中。MP4文件仅可用于快速视觉质量检查（QA）。

## 快速上手
* 查看MCAP文件信息：`mcap info your_sequence.mcap`
* 提取指定话题消息：`mcap cat --topics <topic> your_sequence.mcap > out.bin`
* Python读取工具：执行`pip install mcap`（详见MCAP官方Python文档）或使用任意兼容MCAP格式的工具。常见话题包括RGB、深度图、IMU与标注通道。

## 预期应用场景
* 策略与技能学习（机器人学/视觉语言动作模型（Visual Language Action, VLA））
* 动作检测与分割
* 手部/姿态估计与抓取分析
* 基于深度图的三维重建、同步定位与地图构建（Simultaneous Localization and Mapping, SLAM）、场景理解
* 世界模型预训练与微调

## 定制化服务与专属数据
MicroAGI 可按需提供以下服务：
* 真实场景到仿真环境的转换管线
* 机器学习增强的三维点云与SLAM重建结果
* 通过我们的熟练技工与工厂工人网络采集新数据（价格通常低于市场均价）
* 为您的团队提供设备佩戴与数据处理流水线部署支持

常规交付周期：两周以内（大型项目最长不超过四周）。

## 定制数据订购方式
请发送邮件至`[e-mail]`，并提供以下信息：
* 任务描述
* 所需采集时长或帧数量
* 预估报价
我们将在一个工作日内回复交付周期与最终定价。
咨询问题请联系：`[e-mail]`

## 授权协议
本数据集采用MicroAGI00 非转售开放使用协议v1.0（定制协议），详见[`LICENSE`](./LICENSE)文件。再分发必须以相同协议免费进行。强制引用要求："This work uses the MicroAGI00 dataset (MicroAGI, 2025)."

## 引用提示
若公开使用本数据集或其衍生数据，必须在对应媒介（论文、代码仓库、产品文档、数据集页面、演示说明等）的合理位置添加上述引用语句。对于基于本数据集训练得到的模型或输出，引用为推荐项而非强制要求。