Table 1_Modeling person guessing as a random effect: a Bayesian approach of the two-parameter logistic model.docx

IntroductionGuessing behavior has been an enduring problem that undermines the validity and interpretability of scores from MC items. The present study implements a Bayesian random-effects extension of the 2PLE model which suggests that guessing is a latent individual trait rather than a single item parameter.
MethodsWe implemented a Monte Carlo simulation in a fully crossed design of sample sizes (N = 100–1,000) and test lengths (6–40 items), with 50 replications per condition. Item response data were simulated under the 2PLE model with heterogeneous guessing.
ResultsIn all conditions the estimates of discrimination were larger with the 2PLE than with the 3PL. Gains were especially marked for item difficulty and lower-asymptote estimation that had noticeable distortion under the incorrect 3PL model. Bayesian predictive fit indices (i.e., Leave-One-Out Information Criterion, LOOIC; Widely Applicable Information Criterion, WAIC) consistently supported the 2PLE model under all sample sizes and test lengths. In the proposed framework, the person-level random effect δn reflects differences between individuals in guessing tendency and directly influences the lower asymptote of an item response function.
DiscussionThrough reallocating guessing variance from items to persons the 2PLE random-effects model can better capture diversified response patterns, and obtain a better psychometric performance. Findings are consistent with the conceptualization of guessing as a substantive trait-based process and underscore the utility and necessity of using person-specific guessing models to optimize inferences from test scores.