Multimodal video and IMU kinematic dataset on daily life activities using affordable devices (VIDIMU)

Human activity recognition and clinical biomechanics are challenging problems in physical telerehabilitation medicine. However, most publicly available datasets on human body movements cannot be used to study both problems in an out-of-the-lab movement acquisition setting. The objective of the VIDIMU dataset is to pave the way towards affordable patient tracking solutions for remote daily life activities recognition and kinematic analysis. The VIDIMU dataset includes 54 healthy young adults that were recorded on video and 16 of them were simultaneously recorded using custom IMUs. For each subject, 13 activities were registered using a low-resolution video camera and five Inertial Measurement Units (IMUs). Inertial sensors were placed in the lower or the upper limbs of the subject, respectively for activities that involve movement with the lower or the upper body. Video recordings were postprocessed using the state-of-the-art pose estimator BodyTrack (similar to OpenPose, and included in NVIDIA Maxine-AR-SDK) to provide a sequence of 3D joint positions for each movement. Raw IMU recordings were post-processed to compute joint angles by inverse kinematics with OpenSim. For recordings including simultaneous acquisition of video and IMU data types, these signals were used for data file synchronization. Collected data can be further used in applications related to human activity recognition and biomechanics related experiments in simulated home-like settings.

人体活动识别与临床生物力学均为远程物理康复医学领域中的挑战性研究课题。然而，当前绝大多数公开的人体运动数据集，均无法支持在非实验室运动采集场景下同时开展上述两类问题的研究。
VIDIMU数据集的构建目标，旨在为面向日常远程活动识别与运动学分析的低成本患者追踪方案奠定技术基础。
该数据集共收录54名健康青年受试者的视频采集数据，其中16人同时使用定制惯性测量单元（Inertial Measurement Unit，IMU）完成了同步运动采集。每名受试者需完成13项活动的采集，采集设备包含一台低分辨率摄像机与五台惯性测量单元。惯性传感器分别佩戴于受试者的上下肢，以匹配对应上下肢运动类活动的采集需求。
视频数据后续将通过当前主流的先进姿态估计工具BodyTrack（类似OpenPose，集成于NVIDIA Maxine-AR-SDK中）进行后处理，以生成每项运动的三维关节位置序列。原始IMU采集数据则通过OpenSim软件基于逆运动学算法完成后处理，以计算关节角度。
对于同时采集了视频与IMU数据的受试者，两类信号将被用于实现数据文件的同步对齐。
所采集的数据集可进一步应用于模拟居家场景下的人体活动识别及生物力学相关实验研究。

搜集汇总

背景与挑战

背景概述

VIDIMU数据集是一个多模态数据集，专注于日常活动，使用低成本设备（低分辨率摄像头和IMU）收集数据，旨在支持远程活动识别和运动学分析。数据集包括54名健康年轻成人的视频记录，其中16人同时有IMU数据，覆盖13种活动，并通过后处理提供3D关节位置和关节角度。其特点在于多模态融合、模拟家庭环境设置，适用于人体活动识别和临床生物力学研究。