Kaiwu Dataset

The dataset first provides an integration of human, environment and robot data collection framework with 20 subjects and 30 interaction objects resulting in totally 11,664 instances of integrated actions. For each of the demonstration, hand motions, operation pressures, sounds of the assembling process, multi-view videos, high-precision motion capture information, eye gaze with first-person videos, electromyography signals are all recorded. Fine-grained multi-level annotation based on absolute timestamp, and semantic segmentation labelling are performed. Kaiwu dataset aims to facilitate robot learning, dexterous manipulation, human intention investigation and human-robot collaboration research.本数据集首先构建了人机环境三元数据采集框架，共纳入20名受试者与30个交互物体，最终生成总计11664组集成动作样本。针对每一次演示任务，系统同步采集手部动作、操作压力、装配过程音频、多视角视频、高精度动作捕捉数据、带第一人称视角的眼动追踪数据以及肌电信号。该数据集基于绝对时间戳进行细粒度多级标注，并完成语义分割标记。开悟数据集（Kaiwu Dataset）旨在为机器人学习、灵巧操作、人类意图推断以及人机协作相关研究提供助力。

搜集汇总

背景与挑战

背景概述

Kaiwu Dataset是一个多模态人机交互数据集，包含20名受试者与30个对象交互产生的11,664个集成动作实例，记录了手部动作、操作压力、声音、多视角视频、运动捕捉、眼动及肌电信号等多种数据，并提供基于绝对时间戳的细粒度标注和语义分割标注。该数据集旨在促进机器人学习、灵巧操作、人类意图研究及人机协作等领域的研究。