Longitudinal clinical simulation cohorts

Rotroff Lab
2022-07-01
This repository contains simulated datasets related to Identification of Robust Deep Neural Network Models of Longitudinal Clinical Measurements article published in npj Digital Medicine (2022). These datasets were simulated by manipulating shapes and magnitudes of clinical longitudinal measurements of random blood glucose tests, systolic blood pressure and BMI. Per clinical measurement type, 3168 datasets were generated. In half of these datasets, classes were defined by differences in shape (50%, N=1584) and in the other half, classes were defined by differences in magnitudes (50%, N=1584). The complete list of parameters manipulated is as follows:

Effect Size: 0.25-2.75
Dispersion: 0.25-2.00
Noise: 0%, 10%, 25%, 50%
Irregularity: Moderate (0-3 missing measurements) or High (2-5 missing measurements)

For more information on the simulations, please refer to the Methods section of the paper. Please note that with the inclusion of class imbalance, the total number of cohorts increased to 12,672 in the paper.

Parameter annotations
The parameters are annotated in the file names as follows:

_Simulated_Cohort_EffectSize_Dispersion_Noise_Irregularity_OverlapBetweenClasses.csv
When IrrNMP is used for Irregularity, no irregularity is applied. OverlapBetweenClasses specifies the overlap between the two classes.

Directory Structure
The directory structure for each clinical measurement type is as follows:

bmi_simulation (N=1056)mag_cohorts

shape_cohorts

Simulation_Cohort_IDRandomNumber: Random trajectory base 4

irregularity_cohorts
other_cohorts

Simulation_Cohort_ID_RandomNumber: Random trajectory base 5

irregularity_cohorts
other_cohorts

Simulation_Cohort_ID_RandomNumber: Random trajectory base 6

irregularity_cohorts
other_cohorts

glucose_simulation (1056)mag_cohorts

shape_cohorts

sbp_simulation (N=1056)mag_cohorts

shape_cohorts

Note:
irregularity_cohorts: Effect sizes, dispersions and irregularity were manipulated in these files.

other_cohorts: Effect sizes, dispersions and noise were manipulated in these files.

Please cite the following paper and this dataset in your work:

H. Javidi, et al., Identification of Robust Deep Neural Network Models of Longitudinal Clinical Measurements, Nature Digital Medicine, 2022.