Comparison of real data and simulated data analysis based on the standard error of measurement for stopping algorithm in a computerized adaptive testing: a psychometric study

It aims to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under two stopping rules (SEM 0.3 and 0.25) using both real and simulated data in the medical examination in Korea. This study employed post-hoc simulation and real data analysis to explore the optimal stopping rule for CAT in the medical examination. The real data were obtained from the responses of 3rd-year medical students during examinations in 2020 at Hallym University College of Medicine. Simulated data were generated using estimated parameters from a real item bank in R. Outcome variables included the number of examinees’ passing or failing with SEM values of 0.25 and 0.30, the number of items administered, and the correlation. The accuracy of real CAT data was evaluated by examining classification accuracy based on cut scores of 0.0. The efficiency of all CAT designs was assessed by comparing the average number of items administered under the two stopping rules.