Multimodal Identity Preserved Tracking (MIPT) Dataset

Human behavioral analysis applications in the fields of ambient assisted living (AAL) and human security monitoring require continuous video analysis of individuals. Although intelligent systems deployed in these areas are intended to have a positive impact on the persons involved, subsequent continuous monitoring naturally raises ethical concerns and questions about privacy implications. To address these issues, we present a foundation for identity-preserving 3D human behavior analysis. The dataset is large, at a total of ~85k annotated frames. To reduce privacy intrusion, it consists entirely of spatio-temporally aligned depth and thermal sequences. Annotation is provided as 3D bounding boxes, along with pose labels and consistent person IDs for use in tracking. The dataset is designed to be flexible. Data representation in either image view or point clouds and the option for projected 2D bounding boxes, allows use in a variety of 2D or 3D tasks. Target applications of our work are privacy-sensitive domains that currently require continuous monitoring using RGB-based systems, including ambient assisted living tasks (e.g., motion rehabilitation, fall detection, vital sign detection) and human security monitoring applications, such as construction safety, critical care and correctional facility monitoring.

This database may be used for non-commercial research purpose only. If you publish material based on this database, we request that you include a reference to our paper [1].

[1] T. Heitzinger and M. Kampel “A Foundation for 3D Human Behavior Detection in Privacy-Sensitive Domains”,
in 32nd British Machine Vision Conference (BMVC), 2021