Virtual reality interactions via a user-generic ultrasound human-machine interface for wrist and hand tracking - Example Dataset

Virtual Reality Interactions via a User Generic Ultrasound Human–Machine Interface for Wrist and Hand Tracking This dataset repository accompanies the publication Virtual Reality Interactions via a User Generic Ultrasound Human–Machine Interface for Wrist and Hand Tracking. The repository contains a simple example dataset of simultaneous A-mode Ultrasound and optical motion capture-based joint angles for the wrist (Radial-Ulnar Deviation, Flexion-Extension, Pronation-Supination) and hand (Open-Close of all fingers together). This dataset was structured to be used with the codebase released on GitHub. It contains data from one new participant (not used in the manuscript). The exact same data colection procedures as described in the manuscript were performed for this participant. Data is already processed and made into images to drastically reduce storage requirements and facilitate processing. The image processing pipeline used is described in details on the manuscript. Below we briefly describe its structure. For specific details on how the dataset was collected and how to used it for training predicitive models refer to the publication and code repository. Structure P1S*/ ├── POS*_R*/ │ ├── C*_funct │ ├── ├── images.npy │ ├── ├── labels.npy │ ├── C*_wrist │ ├── ├── images.npy │ ├── ├── labels.npy Where the sub-foldering structure means, `P1S*/` → either `P1S1/` or `P1S2/` and separates data from *Session 1* and *Session 2*. `POS*` → either `POS1`, `POS2`, or `POS3` and separates data from bracelet *Position 1* (most proximal to the elbow), *Position 2*, or *Position 3* (furthest distal to the elbow). `R*` → either `R0`, `R90`, `R180`, or `R270` and separates data from bracelet first placement with a $0^{\circ}$ *Rotation*, a $90^{\circ}$ *Rotation*, a $180^{\circ}$ *Rotation*, or a $270^{\circ}$ *Rotation*. `C*` → either `C1` or `C2` and separates data from both circles of transducers into *Circle 1* (most proximal to the elbow) or *Circle 2* (furthest distal to the elbow). `_funct` and `_wrist` → contains repetitions following the *Functional* video set or the *Wrist* video set. Inside each final sub-folder a single recording can be found separated in two files. `images.npy` → contains a [NumPy](https://numpy.org/) 4D array with the ultrasound data for the specific trial. The array is ordered as $[F, T, C, S]$ where $F = frames$, $T = transmissions$, $C = channels$, $S = samples$. This dataset specificaly has $T=16$, $C=16$, $S=47$ and a variable $F$. `labels.npy` → contains a [NumPy](https://numpy.org/) 2D array with the ultrasound data for the specific trial. The array is ordered as $[F, D]$, where $F = frames$, $D = Degrees~of~Freedom~(DoFs)$. This datadset specificaly has $D = 4$ and a variable $F$. Angle order: 0 = Flexion–Extension, 1 = Ulnar–Radial deviation, 2 = Hand Open–Close, 3 = Pronation–Supination. Files found in the same last sub-folder are synchronized and always have the same $F$. Files can be loaded in Python, for example, by running import numpy as np ultrasound_dataset = np.load("P1S1/POS1_R0/C1_funct/images.npy) labels = np.load("P1S1/POS1_R0/C1_funct/labels.npy) print(ultrasound.shape) # (F, T, C, S) print(labels.shape) # (F, D) License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License CC BY-NC 4.0 Citation If you find this repository helpful, please consider citing: Grandi Sgambato, B., Hodossy, B.K., Barsakcioglu, D.Y. et al. Virtual reality interactions via a user-generic ultrasound human-machine interface for wrist and hand tracking. Nat Commun 16, 11062 (2025). https://doi.org/10.1038/s41467-025-66001-6