A Multimodal Biomechanics Dataset with Synchronized Kinematics and Internal Tissue Motions during Reaching

OverviewThis dataset provides multimodal, time-synchronized measurements collected during slow, rhythmic arm reaching. It bridges internal soft-tissue motion and conventional biomechanical measurements by combining B-mode ultrasound imaging of the upper arm with optical motion capture, surface electromyography, and tri-axial accelerometry.
Data were collected from 36 healthy adult participants across three expertise levels (expert, intermediate, non-expert). Participants performed unconstrained reaching cycles paced by a visual metronome (1 cycle every 6 s) under two hand-orientation conditions (“give”/palm up; “touch”/palm down). All modalities are time-aligned, and the dataset includes processed signals, derived parameters, event annotations, and metadata to support both biomechanics and motor control analyses, as well as machine learning (ML) work on ultrasound tracking.
Key features include:
Synchronized motion capture (upper-limb marker trajectories), ultrasound videos, EMG and accelerometry (palm, biceps, triceps), and derived measures (e.g., reach-cycle events, arm kinematics/speed, tremor events/power, EMG amplitude)B-mode ultrasound videos (60 fps) capturing a transverse view of the triceps/brachialis region during reaching.Frame-level ultrasound point trajectories for 11 tracked points (including humerus, muscle boundary, and within muscle points) spanning ~300,000 frames across the dataset.Citation. Please visit the associated dataset descriptor paper for full details.
[citation]

Ethics & de-identification. Data are de-identified and shared publicly under written informed consent and MIT COUHES approval (Protocol #2201000537).
ContentsThe dataset is organized as follows:
README.txt: high-level documentation and pointers.dataset.csv: one row per participant with demographics/anthropometrics, expertise level, experimental configuration, and paths to the corresponding data files.SHA256SUMS.txt: checksum manifest for integrity verification.data.zip archive (download to obtain all data files in a preserved folder structure), containing:hdf5_files/: 36 (1 per participant) .h5 files with synchronized multimodal timeseries, derived signals, event annotations, and metadata.us_videos/: 36 (1 per participant) ultrasound .mp4 videos (60 fps).hdf5_structure.txt: description of the HDF5 layout (groups/datasets/attributes).exceptions.txt: participant-specific notes/exceptions (e.g., missing channels or irregularities).HDF5 organization (high-level). Each per-participant HDF5 file follows a consistent structure with (i) a metadata group, (ii) an events group (trial/cycle/tremor annotations), and (iii) a timeseries group containing synchronized sensor streams and derived signals, including subgroups with accelerometry, motion capture, electromyography, ultrasound tracker trajectories, tremor-related measures, arm kinematics/speed, and EMG amplitude. See hdf5_structure.txt for details. For the complete definition of every recorded and derived variable (names, descriptions, units, and array shapes), see the manuscript Supplementary Materials, Section HDF5 File Description.

How to useStart with the dataset.csv. Each row corresponds to one participant and includes file paths to the participant’s HDF5 file and ultrasound video.Download and unzip the data.zip archive. The archive preserves the folder structure for hdf5_files/ and us_videos/.Read exceptions.txt before running analyses across all participants (it lists known participant-specific exceptions/notes).Load data using the code repository. Tutorials for loading/visualizing the data and reproducing derived measurements are provided in a companion GitHub repository: https://github.com/RogerPallares/reaching-dataset.Potential use casesSupervised training/benchmarking of ultrasound point-tracking models using the provided tracked point trajectories.Development of ultrasound-based metrics for characterizing soft-tissue motion during movement.Biomechanical and motor-control studies linking internal tissue motion to kinematics, tremor measures, muscle activation, and expertise level.Initial insights on expertise from this dataset are available in:
Namburi, P., Pallarès-López, R., Folgado, D. et al. Efficient elastic tissue motions indicate general motor skill. Sci Rep 15, 36532 (2025). https://doi.org/10.1038/s41598-025-17092-0