Caught in the Act: Predicting Cheating in Unproctored Knowledge Assessment

Cheating is a serious threat in unproctored ability assessment, irrespective of countermeasures taken, anticipated consequences (high vs. low stakes), and test modality (paper-pencil vs. computer-based). In the present study, we examined the power of a) questionnaire-based indicators (i.e., honesty-humility and overclaiming scales), b) test data (i.e., performance with extreme difficult items), and c) para data (i.e., reaction times, switching between browser tabs) to predict participants’ cheating behavior. To this end, 315 participants worked on a knowledge test in an unproctored online assessment and subsequently in a proctored lab assessment. We used multiple regression analysis and an extended latent change score model to assess the potential of the different indicators to predict cheating behavior. In summary, test data and para data were the best predictors of knowledge score differences between the online and lab session, while traditional questionnaire-based indicators, such as scales for honesty or overclaiming, were not predictive. We discuss the findings with respect to unproctored online testing in general and provide practical advice on the detection of cheating in online ability assessments.

在未经监管的能力评估中，作弊行为构成一项严重的威胁，此威胁不受所采取的对策、预期的后果（高与低风险）以及测试形式（纸质笔答与计算机化测试）的影响。在本研究中，我们探讨了以下指标预测参与者作弊行为的能力：a) 基于问卷的指标（即诚信-谦逊和过度声称量表），b) 测试数据（即极端困难题目的表现），以及 c) 辅助数据（即反应时间、在浏览器标签页间切换）。为此，315名参与者在一个未经监管的在线评估中完成了一项知识测试，随后在监管下的实验室评估中进行了测试。我们运用多重回归分析和扩展的潜在变化分数模型来评估不同指标预测作弊行为潜力的可能性。总之，测试数据和辅助数据成为预测在线与实验室测试知识分数差异的最佳预测因子，而传统的基于问卷的指标，如诚信或过度声称的量表，则不具备预测性。我们就一般未经监管的在线测试的结果进行讨论，并提供在在线能力评估中检测作弊行为的实用建议。