[TULIP-Bench Sample] Multi-Camera 3D Pose for Parkinson's Disease Severity Prediction and Treatment Response Detection: Representative Data Sample

This repository provides a representative sample of the TULIP-Bench dataset, released to allow reviewers and researchers to inspect data quality, structure, and formats prior to requesting full access. This sample includes one representative subject per activity type, covering all data modalities released in the full dataset(Fist_Left: Neurips_Sub3, Fist_Right: Neurips_Sub5, Gait_normalPD: Neurips_Sub46, Gait_Neurostimulation: Neurips_DBS_Sub10)

TULIP-Bench is the first publicly available PD movement dataset to jointly provide (1) raw multi-perspective synchronized video, (2) triangulated markerless 3D pose ground truth derived from six hardware-synchronized cameras rather than monocular estimation, (3) both fine motor (bilateral hand movement) and gross motor (gait) tasks in the same cohort, and (4) within-patient treatment recordings with deep brain stimulation (DBS) toggled OFF and ON. No prior public dataset provides all four simultaneously (see Table 1 in the accompanying paper). TULIP-Bench's triangulated multi-camera ground truth enables honest evaluation of monocular and lifting-based 3D pose methods, including the finding that off-the-shelf monocular methods (WHAM, SAM 3D Body).

Recording setup: Six hardware-synchronized Basler cameras (acA1920-155uc, 1920x1200, 80 fps). Layout: Camera 5 (front-facing), Camera 4 (ceiling bird's-eye), Cameras 1, 2, 3, 6 (four elevated corner views). For gait, only Cameras 1, 2, 3, 6 are released; Camera 1 is the designated monocular baseline view. All 6 cameras released for fist activities. Calibrated using OpenCV with a charucoboard pattern.

[Sample Data Structure]* Camera_parameters/   All projection matrices (P = K @ [R|t], shape 3x4) can be used directly for multi-view triangulation and 2D reprojection onto the original video frames.

*Pose3D_Fist_Left_sample_subjects/    sample_subject_pose3d_lefthand.pkl   — Left-hand 3D pose (2400×21×3, 30s x 80fps, real-world mm scale)*Pose3D_Fist_Right_sample_subjects/    sample_subject_pose3d_righthand.pkl  — Right-hand 3D pose (2400×21×3, 30s x 80fps,real-world mm scale)*Pose3D_Gait_normalPD_sample_subject/    sample_subject_pose3d_body.pkl       — Gait 3D body pose (7200×33×3, 90s x 80fps, real-world mm scale)*Pose3D_Gait_Neurostimulation_DBS_subject/    sample_subject_OFF_pose3d_body.pkl   — DBS OFF gait pose (7200×33×3, 90s x 80fps, real-world mm scale)    sample_subject_ON_pose3d_body.pkl    — DBS ON gait pose (7200×33×3, 90s x 80fps, real-world mm scale)

*Video_Fist_Left_sample_subjects/    Camera1.mp4 ... Camera6.mp4          — Face-blurred videos from 6 cameras*Video_Fist_Right_sample_subjects/    Camera1.mp4 ... Camera6.mp4          — Face-blurred videos from 6 cameras*Video_Gait_normal_PD_subject/    Camera1.mp4, Camera2.mp4,    Camera3.mp4, Camera6.mp4             — Face-blurred gait videos, 4 cameras*Video_Gait_Neurostimulation_DBS_subject/        OFF_state/             Camera1.mp4, Camera2.mp4,             Camera3.mp4, Camera6.mp4             — Face-blurred DBS gait videos, 4 cameras        ON_state/             Camera1.mp4, Camera2.mp4,             Camera3.mp4, Camera6.mp4             — Face-blurred DBS gait videos, 4 cameras    (covers both neurostimulation OFF- and ON- states)------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Full dataset (https://doi.org/10.5281/zenodo.20027113) contains:

TULIP-Bench is a multi-camera Parkinson's disease (PD) movement dataset combining synchronized RGB videos and triangulated 3D pose sequences across bilateral fist closure (hand movement, MDS-UPDRS Item 3.5) and gait (MDS-UPDRS Item 3.10) activities, released alongside the paper "Multi-Perspective Video Analysis for Parkinson's Disease Severity Prediction and Treatment Response Detection" (submitted to NeurIPS 2026 Evaluations and Datasets Track).

Cohorts

Observational cohort (normal_PD, N=56 PD patients): Multi-camera recordings of bilateral fist closure (hand movement) and gait at self-selected pace. Each activity has an independent MDS-UPDRS score rated by a licensed movement disorder specialist.

Neurostimulation (Deep Brain Stimulation, DBS) cohort (N=10 subjects, N=20 recordings total for DBS-ON/OFF): Separately recruited patients recorded performing gait in both stimulation OFF (after 10-minute washout) and stimulation ON (after confirmed stable clinical effect) states. Includes per-state UPDRS Gait scores and structured body-part severity annotations (arm swing L/R, stride L/R, elbow flexion L/R).

Dataset structure

Fist_Video.zip: Face-blurred MP4 videos, FistL & FistR, 6 cameras, 56 subjects; 30 seconds with 80fps

Fist_Pose_3D.zip: Triangulated 3D hand poses (2400x21x3, mm; MediaPipe Hand 21-keypoint convention, recommend excluding first 3s to remove hand-raising phase)

Fist_Camera_parameters.zip: Per-subject 3x4 projection matrices (P = K @ [R|t]), Camera 1-6

Fist_labels.csv: MDS-UPDRS Item 3.5 scores (UPDRS_Fist_L, UPDRS_Fist_R)

Gait_Videos.zip: Face-blurred MP4 videos, Gait for normal PD (56 subjects), & Gait_DBS (10 subjects with N=20 recordings total for ON/OFF), 4 cameras-Cameras 1, 2, 3, 6; 90 seconds with 80fps

Gait_Pose_3D.zip: Triangulated 3D body poses (n_framesx17x3, float32, mm; 17-keypoint MediaPipe subset per paper)

Gait_Camera_parameters.zip: Per-subject 3x4 projection matrices, all 6 cameras stored (Cameras 4 and 5 included in pkl but videos not released for gait)

Gait_normalPD_labels.csv: MDS-UPDRS gait scores, for observational cohort

Gait_DBS_labels.csv: MDS-UPDRS and body-part severity labels, for DBS cohort

Access is restricted to academic researchers. Please submit a request via the form below and provide your institutional affiliation and intended use.

IRB approval: Pro00112856 (recording), Pro00116644 (data release).

All participants provided written informed consent.