TULIP Dataset (CVPR 2024): Multi-Camera Videos and Clinician Ratings of the MDS-UPDRS Part III Motor Exam for Parkinson's Disease Assessment
收藏资源简介：
TULIP Dataset (Version 1.0.2) Overview:The TULIP (Three-dimensional Understanding and Learning of Impairments in Parkinson’s) dataset provides high-resolution RGB data from multi-camera setups, supporting research on precision motor assessments for Parkinson’s Disease (PD). Version 1.0.0 features synchronized RGB data from six cameras, capturing multiple angles of PD and healthy participants performing clinically relevant motor tasks. We chose the name TULIP, a nod to the floral emblem of PD research and advocacy, to symbolize our goal for this dataset, to foster transformative new machine learning approaches for PD understanding and treatment. This dataset was published as part of our CVPR 2024 Paper. Kyungdo Kim, Sihan Lyu, Sneha Mantri, Timothy W. Dunn; TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 22551-22562 Data Contents:RGB videos are organized by subject ID and activity, with synchronized recordings from six camera perspectives per activity. Each video documents motor tasks such as gait and finger tapping, in line with the UPDRS standards for PD, allowing for a detailed study of joint angles, tremors, and other movements key to understanding PD progression. · Multi-Camera Video Data: RGB videos from six cameras enable robust 3D pose extraction. · Metadata: Includes camera parameters (intrinsic and extrinsic matrices for 3D reconstruction) and task descriptions. · Clinical Examination Labels: Task-specific labels aligned with clinical motor assessments, with annotations from three clinicians to aid in automated scoring models. We also provided the labels in the csv file format. File Structure:Data is organized by Subject ID > Activities > Camera Perspective, with each folder containing RGB video files. Annotations and activity labels are available in a CSV file for easy correlation of tasks with motor patterns. For the camera parameters (pickle file), each subject has its own set of parameters. When you open the pickle file, the order of the elements is as follows: [proj_matrices, cam_matrices, extrinsic_matrices, rmatrices, rvecs, tvecs, distcoeffs]. Here’s a brief description of each: · proj_matrices: Camera projection matrix (3x4 format) · cam_matrices: Intrinsic camera matrix · extrinsic_matrices: Extrinsic camera matrix · rmatrices: Rotation matrix · rvecs: Rotation vector · tvecs: Translation vector · distcoeffs: Distortion coefficients, which is a zero matrix in our case. Privacy and Consent:Faces are blurred to ensure privacy. This initial version of the dataset contains data from 11 participants, with data from the remaining 4 participants expected to be released in the near future. Code Availability:Behavioral feature extraction demo code for 3D poses will be available on our github (github link can be found on the TULIP Project page). For further details and access to our publication on TULIP data and baseline projects, please visit TULIP Project or CVPR 2024 Paper. Citing this dataset: Please cite our CVPR paper if use this dataset in your work. Citation: Kyungdo Kim, Sihan Lyu, Sneha Mantri, Timothy W. Dunn; TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 22551-22562
TULIP 数据集（版本1.0.2） ## 概述 TULIP（帕金森病损伤三维理解与学习，Three-dimensional Understanding and Learning of Impairments in Parkinson’s）数据集提供多摄像头采集的高分辨率RGB数据，为帕金森病（Parkinson’s Disease, PD）精准运动评估相关研究提供支撑。1.0.0版本包含6个摄像头的同步RGB数据，可捕捉帕金森病患者与健康受试者完成临床相关运动任务的多视角影像。我们将该数据集命名为TULIP，以此致敬帕金森病研究与倡导领域的花卉标识，借以体现本数据集的核心目标：推动面向帕金森病理解与治疗的变革性机器学习新方法发展。 本数据集作为我们发表于CVPR 2024的会议论文的配套数据发布。 作者：Kyungdo Kim、Sihan Lyu、Sneha Mantri、Timothy W. Dunn；论文标题：《TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease》；发表于《IEEE/CVF计算机视觉与模式识别会议（CVPR）论文集》2024年，第22551-22562页。 ## 数据内容 RGB视频按受试者ID与活动类型进行组织，每项活动对应6个摄像头视角的同步录制视频。每个视频记录了步态、手指敲击等符合统一帕金森病评定量表（Unified Parkinson's Disease Rating Scale, UPDRS）标准的运动任务，可用于细致研究关节角度、震颤及其他与帕金森病进展密切相关的运动特征。 · 多摄像头视频数据：6个摄像头采集的RGB视频支持可靠的三维姿态提取。 · 元数据：包含用于三维重建的相机参数（内参、外参矩阵）以及任务描述信息。 · 临床检查标签：与临床运动评估对齐的任务专属标签，由三位临床医师完成标注，以助力自动化评分模型开发。本数据集同时提供CSV格式的标签文件。 ## 文件结构 数据按「受试者ID > 活动类型 > 摄像头视角」的层级结构组织，每个文件夹内包含RGB视频文件。标注与活动标签存储于CSV文件中，便于将任务与运动模式进行关联。相机参数以pickle文件格式存储，每位受试者对应一套独立的参数。打开pickle文件后，元素顺序如下：[proj_matrices, cam_matrices, extrinsic_matrices, rmatrices, rvecs, tvecs, distcoeffs]，各字段简要说明如下： · proj_matrices：相机投影矩阵（格式为3×4） · cam_matrices：相机内参矩阵 · extrinsic_matrices：相机外参矩阵 · rmatrices：旋转矩阵 · rvecs：旋转向量 · tvecs：平移向量 · distcoeffs：畸变系数，本数据集中该系数为零矩阵。 ## 隐私与知情同意 为保护受试者隐私，所有面部影像均已做模糊处理。本初始版本数据集包含11位受试者的数据，剩余4位受试者的数据预计将于近期发布。 ## 代码可用性 用于三维姿态行为特征提取的演示代码将发布于我们的GitHub仓库（GitHub链接可在TULIP项目页面获取）。如需了解更多细节或获取TULIP数据集相关论文与基准项目，请访问TULIP项目页面或CVPR 2024论文页面。 ## 引用该数据集 若您的研究工作中使用了本数据集，请引用我们的CVPR论文。引用格式如下： Kyungdo Kim, Sihan Lyu, Sneha Mantri, Timothy W. Dunn; TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 22551-22562
