Data Analyses and Sample Size Planning for Intensive Longitudinal Intervention Studies with Dynamic Structural Equation Modeling

Intensive longitudinal interventions (ILIs) have gained prominence as powerful tools for treating and preventing mental and behavioral disorders (Heron & Smyth, 2010). However, most studies analyze ILI data use traditional methods like ANOVA or linear mixed models, which overlook individual differences and the autocorrelation structure inherent in time series data (Hedeker et al., 2008). Moreover, existing methods typically assess intervention effects based solely on changes in the mean level of key variables (e.g., anxiety). This study demonstrates how to model ILI data within the framework of dynamic structural equation modeling (DSEM) to evaluate intervention effects across three dimensions: mean, autoregression, and individual intra-variation (IIV), for two intervention designs: non-randomized single-arm trial (NST) and randomized control trial (RCT). We conducted two simulation studies to investigate sample size recommendations for DSEM in ILI studies, considering both statistical power and accuracy in parameter estimation (AIPE). Additionally, we compared the two designs based on type I error rate in a separate simulation. Finally, we illustrated sample size planning using data from a pre-ILI study focused on reducing appearance anxiety.Simulation Studies 1 and 2 investigated the power and AIPE across varying sample sizes, as well as the required sample size for both NST and RCT designs. The effect sizes of intervention effects for mean, autoregression and IIV were fixed at the medium level. Two factors regarding sample size were manipulated: number of participants (N = 30, 60, 100,150, 200, 300,400), number of time-points (T= 10, 20, 40, 60, 80, 100). The data-generating models and fitted models were identical, with analysis conducted using Mplus 8.10 and Bayesian estimation. Model performance was assessed in terms of convergence rate, power and AIPE for intervention effects, as well as bias in the standard errors of the intervention effects. Simulation Study 3 assessed the type I error rate for both designs when changes in the control group was different from zero, indicating a change (on average) due to time. Last, the empirical study conducted sample size planning based on a pre-study aimed at reducing appearance anxiety using an ILI design.The results are as following. First, there were no convergence issues under all the conditions. Second, power increased, width of the credible intervals decreased as either N or T increased. However, a minimum of 60 participants was required to achieve adequate power (i.e., ). The relative bias in intervention effect was generally small. Except in the NST design, the intervention effects on autoregression and IIV were underestimated when the number of time-points was low (i.e., T=10 or 20), while in the RCT design, the intervention effect on mean was underestimated when sample size in both levels were small (i.e., N=30 or 60, T=10). Bias in the standard error was also minimal across conditions. Third, a credible interval width contours plot could be applied to recommend sample sizes in DSEM. The sample size requirements based on power and AIPE were different under NST design and RCT design, with RCT requiring larger samples due to the addition of a control group. Fourth, when a natural change (on average) occurred between pre- and post-intervention phrases, the NST design led to inflated type I error rates compared to the RCT design, particularly with larger sample sizes.In conclusion, we first recommend using DSEM to analyze ILI data, as it better captures intervention effects on mean, autoregression, and IIV. Second, practitioners should select either the NST or RCT design based on theoretical and empirical considerations. While the RCT design controls for confounding factors like time-related changes in mean, it requires a larger sample size. NST designs were usually conducted before large RCTs with relatively small samples, especially for rare participants. Finally, choosing the true parameters for the data-generating model was crucial in sample size planning using a monte carlo method. We suggested derive these parameters from pre-studies, similar empirical studies or meta-analysis when possible, as many parameters (i.e., regarding to fixed effects and random effects) should be set in DSEM. If no prior information is available, we suggest following the procedures outlined in this study.This database includes the code for data generating and analysis in simulation studies, and data, code and results in empirical example.