Kaiwu

The dataset first provides an integration of human, environment and robot data collection framework with 20 subjects and 30 interaction objects resulting in totally 11,664 instances of integrated actions. For each of the demonstration, hand motions, operation pressures, sounds of the assembling process, multi-view videos, high-precision motion capture information, eye gaze with first-person videos, electromyography signals are all recorded. Fine-grained multi-level annotation based on absolute timestamp, and semantic segmentation labelling are performed. Kaiwu dataset aims to facilitate robot learning, dexterous manipulation, human intention investigation and human-robot collaboration research.

本数据集首先搭建了人机环一体化数据采集框架，共纳入20名受试者与30个交互对象，最终生成总计11664条集成动作样本。针对每一次示教演示，系统同步采集手部运动、操作压力、装配过程音频、多视角视频、高精度运动捕捉数据、搭载第一人称视角视频的眼动追踪数据以及肌电信号。同时基于绝对时间戳完成细粒度多级别标注，并开展语义分割标注工作。开物数据集旨在为机器人学习、灵巧操作、人类意图推断以及人机协作相关研究提供支撑。

搜集汇总

背景与挑战

背景概述

Kaiwu数据集是一个多模态数据集，集成了人类、环境和机器人交互数据，包含11,664个动作实例，记录了手部动作、声音、视频、眼动等多种信息，并进行了细粒度标注。该数据集旨在支持机器人学习、灵巧操作和人机协作研究。