[TULIP-Bench Sample] Multi-perspective videos and 3D poses capturing multi-task Parkinson's disease assessments and deep brain stimulation treatment effects: Representative Data Sample

Notice for Authorized Reviewers — Private Access LinkThis private access link has been generated solely for the purpose of reviewing this academic submission to NeurIPS 2026. The record is publicly accessible, but files are restricted to users with access. You are receiving private access links to enable review of the dataset for academic purposes only. Only access the dataset if you (1) agree to use the dataset only for the purpose of reviewing this academic product, and (2) accept the following license agreement. By downloading any file from this repository using this link, you agree to the following conditions, which constitute a legally binding data use agreement : You will use the data exclusively for the purpose of reviewing this academic submission and for no other purpose. You will not redistribute, share, publish, or transmit this link or any downloaded files to any third party. You will delete all downloaded files upon completion of the review process. You will not attempt to re-identify any individuals in the dataset. You will not use the data for commercial purposes. This dataset, and any copyrights therein, are owned by Duke University. You agree to an open license under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BYNC-ND 4.0) license: https://creativecommons.org/licenses/by-nc-nd/4.0/. Outside contributions to the Duke-owned dataset cannot be accepted unless the contributor assigns copyrights to any modifications, changes, and/or derivatives over to Duke University. Please note that this dataset is distributed AS IS, WITHOUT ANY WARRANTY; and without the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Downloading any file constitutes your acceptance of these terms. Note on file access: This repository uses Zenodo's restricted-access system to comply with our data use agreement. You may see the message "The record is publicly accessible, but files are restricted to users with access" when visiting the page; this is expected behavior and does not mean your access has been denied. If you have received a private access link, you can download all files directly using that link without creating a Zenodo account or logging in. If you encounter persistent access issues, please contact us at [e-mail]. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ This repository provides a representative sample of the TULIP-Bench dataset, released to allow reviewers and researchers to inspect data quality, structure, and formats prior to requesting full access. This sample includes one representative subject per activity type, covering all data modalities released in the full dataset(Fist_Left: Neurips_Sub3, Fist_Right: Neurips_Sub5, Gait_normalPD: Neurips_Sub46, Gait_Neurostimulation: Neurips_DBS_Sub10) TULIP-Bench is the first publicly available PD movement dataset to jointly provide (1) raw multi-perspective synchronized video, (2) triangulated markerless 3D pose ground truth derived from six hardware-synchronized cameras rather than monocular estimation, (3) both fine motor (bilateral hand movement) and gross motor (gait) tasks in the same cohort, and (4) within-patient treatment recordings with deep brain stimulation (DBS) toggled OFF and ON. No prior public dataset provides all four simultaneously (see Table 1 in the accompanying paper). TULIP-Bench's triangulated multi-camera ground truth enables honest evaluation of monocular and lifting-based 3D pose methods, including the finding that off-the-shelf monocular methods (WHAM, SAM 3D Body). Recording setup: Six hardware-synchronized Basler cameras (acA1920-155uc, 1920x1200, 80 fps). Layout: Camera 5 (front-facing), Camera 4 (ceiling bird's-eye), Cameras 1, 2, 3, 6 (four elevated corner views). For gait, only Cameras 1, 2, 3, 6 are released; Camera 1 is the designated monocular baseline view. All 6 cameras released for fist activities. Calibrated using OpenCV with a charucoboard pattern. [Sample Data Structure]* Camera_parameters/ All projection matrices (P = K @ [R|t], shape 3x4) can be used directly for multi-view triangulation and 2D reprojection onto the original video frames. *Pose3D_Fist_Left_sample_subjects/ sample_subject_pose3d_lefthand.pkl — Left-hand 3D pose (2400×21×3, 30s x 80fps, real-world mm scale)*Pose3D_Fist_Right_sample_subjects/ sample_subject_pose3d_righthand.pkl — Right-hand 3D pose (2400×21×3, 30s x 80fps,real-world mm scale)*Pose3D_Gait_normalPD_sample_subject/ sample_subject_pose3d_body.pkl — Gait 3D body pose (7200×33×3, 90s x 80fps, real-world mm scale)*Pose3D_Gait_Neurostimulation_DBS_subject/ sample_subject_OFF_pose3d_body.pkl — DBS OFF gait pose (7200×33×3, 90s x 80fps, real-world mm scale) sample_subject_ON_pose3d_body.pkl — DBS ON gait pose (7200×33×3, 90s x 80fps, real-world mm scale) *Video_Fist_Left_sample_subjects/ Camera1.mp4 ... Camera6.mp4 — Face-blurred videos from 6 cameras*Video_Fist_Right_sample_subjects/ Camera1.mp4 ... Camera6.mp4 — Face-blurred videos from 6 cameras*Video_Gait_normal_PD_subject/ Camera1.mp4, Camera2.mp4, Camera3.mp4, Camera6.mp4 — Face-blurred gait videos, 4 cameras*Video_Gait_Neurostimulation_DBS_subject/ OFF_state/ Camera1.mp4, Camera2.mp4, Camera3.mp4, Camera6.mp4 — Face-blurred DBS gait videos, 4 cameras ON_state/ Camera1.mp4, Camera2.mp4, Camera3.mp4, Camera6.mp4 — Face-blurred DBS gait videos, 4 cameras (covers both neurostimulation OFF- and ON- states)------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Full dataset (https://doi.org/10.5281/zenodo.20027113) contains: TULIP-Bench is a multi-camera Parkinson's disease (PD) movement dataset combining synchronized RGB videos and triangulated 3D pose sequences across bilateral fist closure (hand movement, MDS-UPDRS Item 3.5) and gait (MDS-UPDRS Item 3.10) activities, released alongside the paper "Multi-Perspective Video Analysis for Parkinson's Disease Severity Prediction and Treatment Response Detection" (submitted to NeurIPS 2026 Evaluations and Datasets Track). Cohorts Observational cohort (normal_PD, N=56 PD patients): Multi-camera recordings of bilateral fist closure (hand movement) and gait at self-selected pace. Each activity has an independent MDS-UPDRS score rated by a licensed movement disorder specialist. Neurostimulation (Deep Brain Stimulation, DBS) cohort (N=10 subjects, N=20 recordings total for DBS-ON/OFF): Separately recruited patients recorded performing gait in both stimulation OFF (after 10-minute washout) and stimulation ON (after confirmed stable clinical effect) states. Includes per-state UPDRS Gait scores and structured body-part severity annotations (arm swing L/R, stride L/R, elbow flexion L/R). Dataset structure Fist_Video.zip: Face-blurred MP4 videos, FistL & FistR, 6 cameras, 56 subjects; 30 seconds with 80fps Fist_Pose_3D.zip: Triangulated 3D hand poses (2400x21x3, mm; MediaPipe Hand 21-keypoint convention, recommend excluding first 3s to remove hand-raising phase) Fist_Camera_parameters.zip: Per-subject 3x4 projection matrices (P = K @ [R|t]), Camera 1-6 Fist_labels.csv: MDS-UPDRS Item 3.5 scores (UPDRS_Fist_L, UPDRS_Fist_R) Gait_Videos.zip: Face-blurred MP4 videos, Gait for normal PD (56 subjects), & Gait_DBS (10 subjects with N=20 recordings total for ON/OFF), 4 cameras-Cameras 1, 2, 3, 6; 90 seconds with 80fps Gait_Pose_3D.zip: Triangulated 3D body poses (7200x33x3, mm; 17-keypoint MediaPipe subset per paper) Gait_Camera_parameters.zip: Per-subject 3x4 projection matrices, all 6 cameras stored (Cameras 4 and 5 included in pkl but videos not released for gait) Gait_normalPD_labels.csv: MDS-UPDRS gait scores, for observational cohort Gait_DBS_labels.csv: MDS-UPDRS and body-part severity labels, for DBS cohort Access is restricted to academic researchers. Please submit a request via the form below and provide your institutional affiliation and intended use. IRB approval: Pro00112856 and Pro00116644. All participants provided written informed consent.致授权审稿人：私人访问链接 本私人访问链接仅为审核本次提交至NeurIPS 2026的学术稿件而生成。本数据集记录公开可见，但文件仅授权指定用户访问。您收到本私人访问链接仅用于学术目的的数据集审核。仅当您满足以下两项要求时方可访问本数据集：(1) 同意仅将本数据集用于本次学术稿件的审核工作；(2) 接受下述许可协议。通过本链接下载本仓库中的任何文件，即表示您同意下述条款，该条款构成具有法律约束力的数据使用协议： 1. 您仅可将本数据集用于本次学术稿件的审核工作，不得用于其他任何用途。 2. 您不得向任何第三方重新分发、共享、发布或传输本链接或任何已下载的数据集文件。 3. 审核流程结束后，请删除所有已下载的数据集文件。 4. 您不得尝试重新识别数据集中的任何个体。 5. 您不得将本数据集用于商业用途。 本数据集及其中的所有版权归杜克大学（Duke University）所有。您同意遵循知识共享署名-非商业性使用-禁止演绎4.0国际许可协议（Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International，CC BY-NC-ND 4.0），许可链接为：https://creativecommons.org/licenses/by-nc-nd/4.0/。除非贡献者将其对杜克大学所有数据集的修改、变更和/或衍生作品的版权转让给杜克大学，否则不予接受外部对本数据集的贡献。请注意，本数据集按“现状”提供，不提供任何明示或暗示的担保，包括但不限于适销性或特定用途适用性的默示担保。 通过下载本仓库中的任何文件，即表示您接受上述条款。 文件访问说明：本仓库使用Zenodo的受限访问系统以符合我们的数据使用协议。访问页面时可能会看到提示"本数据集记录公开可见，但文件仅授权指定用户访问"，此为预期行为，并不代表您的访问权限未被授予。若您已收到私人访问链接，可直接通过该链接下载所有文件，无需创建Zenodo账户或登录。若持续遇到访问问题，请发送邮件至[e-mail]联系我们。 ———————————————————————————————————————————————————————————————————————————————————————————————————— 本仓库提供TULIP-Bench数据集的代表性样本，旨在供审稿人与研究人员在申请完整数据集访问权限前，检查数据质量、结构与格式。本样本包含每种活动类型的一个代表性受试者，涵盖完整数据集发布的所有数据模态（左手握拳：Neurips_Sub3，右手握拳：Neurips_Sub5，常规帕金森病（Parkinson's Disease, PD）步态：Neurips_Sub46，神经刺激下步态：Neurips_DBS_Sub10） TULIP-Bench是首个公开可用的帕金森病运动数据集，可同时提供以下四类内容：(1) 原始多视角同步视频；(2) 由六台硬件同步相机推导得到的三角化无标记3D姿态真值，而非单目估计结果；(3) 同一队列中同时包含精细运动（双侧手部运动）与粗大运动（步态）任务；(4) 患者在脑深部刺激（Deep Brain Stimulation, DBS）开启与关闭状态下的治疗记录。目前尚无公开数据集可同时提供全部四类内容（详见配套论文中的表1）。TULIP-Bench的三角化多相机真值可用于公正评估单目及基于提升法的3D姿态估计方法，包括验证现有商用单目方法（WHAM、SAM 3D Body）的性能表现。 录制设置：六台硬件同步的Basler相机（型号acA1920-155uc，分辨率1920×1200，帧率80 fps）。相机布局：相机5（正前方视角）、相机4（顶部鸟瞰视角）、相机1、2、3、6（四个高位角落视角）。步态任务仅发布相机1、2、3、6的录制内容，其中相机1为指定的单目基准视角。握拳动作任务则发布全部6台相机的录制内容。使用Charuco棋盘格图案通过OpenCV完成相机标定。 ### 样本数据结构 * 相机参数文件夹（Camera_parameters/）：所有投影矩阵（P = K @ [R|t]，形状为3×4）可直接用于多视图三角化与2D重投影至原始视频帧。 * 左手握拳样本受试者3D姿态文件夹（Pose3D_Fist_Left_sample_subjects/）：sample_subject_pose3d_lefthand.pkl —— 左手3D姿态（2400×21×3，对应30秒×80 fps，单位为真实世界毫米） * 右手握拳样本受试者3D姿态文件夹（Pose3D_Fist_Right_sample_subjects/）：sample_subject_pose3d_righthand.pkl —— 右手3D姿态（2400×21×3，对应30秒×80 fps，单位为真实世界毫米） * 常规帕金森步态样本受试者3D姿态文件夹（Pose3D_Gait_normalPD_sample_subject/）：sample_subject_pose3d_body.pkl —— 步态全身3D姿态（7200×33×3，对应90秒×80 fps，单位为真实世界毫米） * 神经刺激下DBS步态样本受试者3D姿态文件夹（Pose3D_Gait_Neurostimulation_DBS_subject/）： sample_subject_OFF_pose3d_body.pkl —— DBS关闭状态下的步态姿态（7200×33×3，对应90秒×80 fps，单位为真实世界毫米） sample_subject_ON_pose3d_body.pkl —— DBS开启状态下的步态姿态（7200×33×3，对应90秒×80 fps，单位为真实世界毫米） * 左手握拳样本受试者视频文件夹（Video_Fist_Left_sample_subjects/）：Camera1.mp4 … Camera6.mp4 —— 6台相机拍摄的面部模糊处理视频 * 右手握拳样本受试者视频文件夹（Video_Fist_Right_sample_subjects/）：Camera1.mp4 … Camera6.mp4 —— 6台相机拍摄的面部模糊处理视频 * 常规帕金森步态受试者视频文件夹（Video_Gait_normal_PD_subject/）：Camera1.mp4、Camera2.mp4、Camera3.mp4、Camera6.mp4 —— 4台相机拍摄的面部模糊处理步态视频 * 神经刺激下DBS步态受试者视频文件夹（Video_Gait_Neurostimulation_DBS_subject/）： OFF_state/：Camera1.mp4、Camera2.mp4、Camera3.mp4、Camera6.mp4 —— 4台相机拍摄的面部模糊处理DBS关闭状态步态视频 ON_state/：Camera1.mp4、Camera2.mp4、Camera3.mp4、Camera6.mp4 —— 4台相机拍摄的面部模糊处理DBS开启状态步态视频 （涵盖神经刺激关闭与开启两种状态） ———————————————————————————————————————————————————————————————————————————————————————————————————— 完整数据集（https://doi.org/10.5281/zenodo.20027113）包含： TULIP-Bench是一个多视角帕金森病运动数据集，整合了同步RGB视频与三角化3D姿态序列，涵盖双侧握拳（手部运动，对应运动障碍学会统一帕金森病评定量表（Movement Disorder Society-Unified Parkinson's Disease Rating Scale, MDS-UPDRS）项目3.5）与步态（对应MDS-UPDRS项目3.10）两类活动，随提交至NeurIPS 2026评估与数据集赛道的论文《多视角视频分析用于帕金森病严重程度预测与治疗响应检测》一同发布。 ### 受试队列 1. 观察队列（常规帕金森组，N=56名帕金森患者）：记录双侧握拳（手部运动）与自定步速步态的多视角视频。每项活动均由持证运动障碍专科医师评分，获得独立的MDS-UPDRS评分。 2. 神经刺激（脑深部刺激，DBS）队列（N=10名受试者，共20次录制，对应DBS开启/关闭状态）：招募的患者分别在刺激关闭状态（经过10分钟洗脱期）与刺激开启状态（临床效果确认稳定后）完成步态录制。包含每种状态下的UPDRS步态评分与结构化肢体严重程度注释（左侧/右侧手臂摆动、左侧/右侧步幅、左侧/右侧肘关节屈曲）。 ### 数据集结构 * Fist_Video.zip：面部模糊处理的MP4视频，包含左侧握拳与右侧握拳内容，对应6台相机、56名受试者，时长30秒，帧率80 fps * Fist_Pose_3D.zip：三角化3D手部姿态（2400×21×3，单位为毫米；采用MediaPipe手部21个关键点标注规范，建议排除前3秒以移除抬手阶段） * Fist_Camera_parameters.zip：每名受试者的3×4投影矩阵（P = K @ [R|t]），对应相机1-6 * Fist_labels.csv：MDS-UPDRS项目3.5评分（包含UPDRS_Fist_L、UPDRS_Fist_R字段） * Gait_Videos.zip：面部模糊处理的MP4视频，包含常规帕金森步态（56名受试者）与DBS步态（10名受试者，共20次开启/关闭状态录制），对应4台相机：相机1、2、3、6，时长90秒，帧率80 fps * Gait_Pose_3D.zip：三角化3D全身姿态（7200×33×3，单位为毫米；采用论文中指定的17个关键点MediaPipe子集） * Gait_Camera_parameters.zip：每名受试者的3×4投影矩阵，存储全部6台相机的参数（步态任务未发布相机4与5的视频，但pkl文件中包含这两台相机的参数） * Gait_normalPD_labels.csv：观察队列的MDS-UPDRS步态评分 * Gait_DBS_labels.csv：DBS队列的MDS-UPDRS评分与肢体严重程度标签 本数据集仅对学术研究人员开放访问。请通过下方表单提交访问申请，并提供您的机构隶属关系与预期使用用途。 伦理审查批准号：Pro00112856与Pro00116644。 所有受试者均已签署书面知情同意书。