UNIPD-BPE: Synchronized RGB-D and Inertial Data forMultimodal Body Pose Estimation and Tracking

The ability to estimate human motion without requiring any external on-body sensor or marker is of paramount importance in a variety of fields, ranging from human-robot interaction, industry 4.0, surveillance, and telerehabilitation.The recent development of portable low-cost RGB-D cameras allowed to push forward the accuracy of markerless motion capture systems.However, despite the widespread use of such sensors, a dataset including complex scenes with multiple interacting people, recorded with a calibrated network of RGB-D cameras and an external system for assessing the pose estimation accuracy is still missing.In this paper, we present UNIPD-BPE, an extensive dataset containing both single-person and multi-person sequences with up to 4 interacting people.A network with 5 Microsoft Azure Kinect cameras is exploited to record synchronized high definition RGB and depth data of the scene from multiple viewpoints, as well as to estimate the subjects' poses using the Azure Kinect Body Tracking SDK.Simultaneously, full-body Xsens MVN Awinda inertial suits allow to obtain accurate ground truth poses and anatomical joint angles, while also providing raw data of the 17 IMUs required by the system.This dataset aims to push forward the development and validation of multi-camera markerless body pose estimation and tracking algorithms, as well as multimodal approaches focused on merging visual and inertial data. This is just a sample. The full dataset can be accessed at the following OneDrive link.