Sole-HARmony: A long-duration, free-living, multimodal dataset with frame-level ground-truth annotations for real-world human activity recognition

We present Sole-HARmony (Sole-based Human Activity Recognition via Multimodal Observation of Naturalistic mobilitY), a fully annotated, free-living human activity recognition (HAR) dataset collected using multimodal instrumented insoles, each equipped with a 6-degree-of-freedom IMU and an eight-cell FSR array, supplemented by a downward-facing body-worn camera for continuous activity recording over extended periods. While continuous monitoring of daily-life ambulatory activities can provide clinically meaningful insights, most HAR research still relies on data collected in controlled or semi-controlled environments. As a result, models trained on in-lab datasets often show reduced accuracy when applied to unstructured, free-living data. Publicly available free-living HAR datasets remain scarce, and existing ones are frequently limited to coarse activity labels (e.g., self-report or sparse image/video-based annotation) and unimodal sensing, leaving the potential benefits of multimodal configurations largely unexplored. The few available multimodal HAR datasets are typically limited to short, single-day recordings per participant, often totaling no more than ~30 hours. This highlights the need for a long-duration, fully annotated, multimodal dataset for real-world HAR applications.The dataset includes recordings from 13 participants, each collecting 4–5 hours of free-living data per day over five consecutive days while performing unscripted daily activities without supervision, for a total of 286.75 hours of recording. Participants were equipped with a pair of instrumented insoles and a waist-mounted body-worn camera to provide continuous ground-truth annotation. Additionally, a subset of 10 participants wore a validated thigh-mounted activity monitor (activPAL3 micro), enabling direct comparison with a well-established research-grade system. Overall, this dataset provides a valuable resource for the wearable sensing and HAR communities, particularly for the development and evaluation of algorithms in realistic, multi-day, unscripted environments. By combining long-duration recordings, fine-grained annotations, and multimodal sensing, it helps bridge the gap between laboratory-based model development and deployment-ready HAR systems.