Opportunity++: A Multimodal Dataset forVideo- and Wearable, Object and AmbientSensors-based Human Activity Recognition

Opportunity++ is a precisely annotated dataset designed to support AI and machine learning research focused on the multimodal perception and learning of human activities (e.g. short actions, gestures, modes of locomotion, higher-level behavior). The Opportunity++ dataset is a significant multimodal extension of the original OPPORTUNITY Activity Recognition Dataset available at https://archive.ics.uci.edu/ml/datasets/OPPORTUNITY+Activity+Recognition. Opportunity++ includes the original video recordings as well as video-derived skeleton tracking data. This enables a wide-range of novel multimodal activity recognition research based on video data, ambient- and object-integrated sensors and wearable sensors (classification, automatic data segmentation, sensor fusion, feature extraction, etc).This release includes:Body-worn sensors: 7 inertial measurement units, 12 3D acceleration sensors, 4 3D localization informationObject sensors: 12 objects with 3D acceleration and 2D rate of turnAmbient sensors: 13 switches and 8 3D acceleration sensorsNewly released anonymized side-view videosNewly released OpenPose tracks for all the people in the videos. This includes the coordinates of the joints (nose, neck, …) of all the users in the video frames.The dataset included data from 4 users performing everyday living activities in a kitchen environment. For each user the dataset includes 6 different runs. Five runs, termed Activity of Daily Living (ADL), followed a given scenario as detailed below. The sixth termed Drill Run, was designed to generate a large number of activity instances in a more constrained scenario. The ADL run consists of temporally unfolding situations. In each situation (e.g. preparing sandwich), a large number of action primitives occur (e.g. reach for bread, move to bread cutter, operate bread cutter).The dataset includes a total of 19.75 hours of sensor data annotated with multiple tracks: 1.88 hours of actions performed with any of the two hands, 6.01 hours of locomotion status, 3.02 hours of annotated data for action performed with a specific hand and 4.89 hours of high level activities. Moreover, the sensors placed on the objects produced a total of 3.92 hours of annotated data.Overall, the dataset comprise of more than 24000 unique annotations, divided in 2551 activity instances for \texttt{ML both arm}, 3653 activity instances of locomotion, 12242 action instances performed with a single specific hand, 122 instances of high level activities and 6103 instances of interaction with the object in the kitchen.

Opportunity++是一款经过精准标注的数据集，旨在支撑面向人类活动（如短时动作、手势、移动模式、高阶行为）的多模态感知与学习相关的人工智能（Artificial Intelligence, AI）和机器学习（Machine Learning, ML）研究。
Opportunity++数据集是原始OPPORTUNITY活动识别数据集的重要多模态扩展版本，原始数据集可从https://archive.ics.uci.edu/ml/datasets/OPPORTUNITY+Activity+Recognition获取。
Opportunity++包含原始视频录像以及源自视频的骨骼追踪数据，可支撑基于视频数据、环境与物体集成传感器以及可穿戴传感器的各类新型多模态活动识别研究，涵盖分类、自动数据分割、传感器融合、特征提取等研究方向。

本次发布包含以下内容：
可穿戴传感器：7个惯性测量单元（Inertial Measurement Unit, IMU）、12个三维加速度传感器、4组三维定位信息
物体传感器：12个搭载三维加速度与二维角速度传感器的物体
环境传感器：13个开关传感器与8个三维加速度传感器
全新发布的匿名侧视角视频
全新发布的视频中所有人物的OpenPose骨骼追踪结果，涵盖视频帧内所有受试者的关节（鼻尖、颈部等）坐标。

该数据集收录了4名受试者在厨房环境中完成日常起居活动的相关数据。每位受试者对应6组独立录制序列：其中5组为日常生活活动（Activity of Daily Living, ADL）序列，遵循下述给定的场景流程；第6组为专项演练序列（Drill Run），旨在在更受限的场景下生成大量活动样本。
ADL序列由随时间推进的多个场景构成，每个场景（例如制作三明治）中会包含大量动作原语（例如伸手取面包、移动至面包切片机、操作面包切片机）。

该数据集总计包含19.75小时的标注传感器数据，涵盖多类标注轨道：1.88小时的双手任意一侧动作数据、6.01小时的移动状态数据、3.02小时的单侧手部动作标注数据，以及4.89小时的高阶活动数据。此外，部署在物体上的传感器总计产生了3.92小时的标注数据。

整体而言，该数据集包含超过24000条独立标注条目，具体划分为：2551条双臂活动实例、3653条移动活动实例、12242条单侧手部动作实例、122条高阶活动实例，以及6103条厨房物体交互实例。

搜集汇总

背景与挑战

背景概述

Opportunity++是一个多模态数据集，专为支持人类活动识别的AI和机器学习研究设计，扩展了原始OPPORTUNITY数据集，新增视频和骨架跟踪数据。它包含身体穿戴、物体和环境传感器数据，来自4名用户在厨房环境中的日常活动，总共有19.75小时标注数据，覆盖超过24000个活动实例，适用于多模态感知和传感器融合研究。