Hypothesis Testing Methods for Multivariate Multi-Occasion Intra-Individual Change

In psychological and educational measurement, it is often of interest to assess change in an individual. The current study expanded on previous research by introducing methods that can evaluate individual change on multiple latent traits measured on multiple occasions. The four methods considered are the likelihood ratio test (LRT), the multivariate Wald test (MWT), the modified multivariate Wald test (MMWT), and the score test (ST). Simulation studies were conducted to examine the true positive rate (TPR) and the false positive rate (FPR) of the new methods under a conventional fixed-form test and a computerized adaptive test (CAT). Manipulated variables included the number of occasions, change magnitudes, patterns of change, and correlations between latent traits. Results revealed that, in terms of FPR, all methods except MWT had close adherence to the nominal significance level. Among the three methods, the LRT is recommended as it provided a balance between FPR and TPR. Larger change magnitude yielded higher TPR, regardless of the remaining factors. With the same test length, a CAT yielded higher TPR than a conventional test. Real-data examples are provided of identifying psychometrically significant change across two to four occasions using a multivariate adaptive self-report medical outcomes measure from hospitalized patients. The detection of significant change among the three methods agreed highly, and those patients identified as having significant change exhibited large profile differences, which provided support for the valid performance of the proposed methods.