Analyzing Transitions in Sequential Data with Marginal Models

Various areas of educational research are interested in the transitions between different states—or events—in sequential data, with the goal of understanding the significance of these transitions; one notable exampleis affect dynamics, which aims to identify important transitions between affective states. Unfortunately,several works have uncovered issues with the metrics and procedures commonly used to analyzethese transitions. As such, our goal in this work is to address these issues by outlining an alternativeprocedure that is based on the use of marginal models. We begin by looking at the specific mechanismsresponsible for a recently discovered statistical bias with several metrics used in sequential data analysis.After giving a theoretical explanation for the issue, we show that the marginal model procedure appearsto adjust for this bias. Next, a related problem is that the common practice of removing transitions torepeated states has been shown to have unintended side-effects—to account for this issue, we developa method for extending the marginal model procedure to this specific type of analysis. Finally, in arecent study evaluating the problem of multiple comparisons and sequential data analysis, the Benjamini-Hochberg (BH) procedure, a commonly used approach to control for false discoveries, did not performas expected. By applying a technique from the biostatistics and epidemiology literature, we show that theperformance of the BH procedure, when used with the marginal model method, can be brought back to itsexpected level. In all of our analyses, we evaluate the proposed method by both running simulations andusing actual student data. The results indicate that the marginal model procedure seemingly compensatesfor the problems observed with other transition metrics, thus resulting in more accurate estimates of theimportance of transitions between states.