Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of measurement, and cognitive load, using a quasi-experimental crossover design. Biomedical students were randomized into two groups to sit a 65-item formative exam starting with the MCQ-CBA followed by the Proxy-CBA (group 1, n = 38), or the reverse (group 2, n = 35). Subsequently, a questionnaire on perceived cognitive load was taken, answered by 71 participants. Both CBA formats were analyzed according to parameters of the Classical Test Theory and the Rasch model. Compared to the MCQ-CBA, the Proxy-CBA had lower raw scores (p 2 = 0.276), higher reliability estimates (p 2 = 0.498), lower SEM estimates (p 2 = 0.807), and lower theta ability scores (p 2 = 0.288). The questionnaire revealed no significant differences between both CBA tests regarding perceived cognitive load. Compared to the MCQ-CBA, the Proxy-CBA showed increased reliability and a higher degree of validity with similar cognitive load, suggesting its utility as an alternative assessment format.多项选择题（Multiple Choice Questions，MCQs）存在提示线索效应、题目质量欠佳以及仅能考察事实性知识等固有局限。本研究提出一种采用计算机化测评（Computer-Based Assessment，CBA）形式的新型多模态测评工具，其包含多种替代题型，将该工具命名为Proxy-CBA。本研究采用准实验交叉设计，将Proxy-CBA与标准MCQ-CBA从效度、信度、测量标准误（Standard Error of Measurement，SEM）以及认知负荷四个维度开展对比评估。研究招募生物医学专业学生，将其随机分为两组：第一组（n=38）先完成包含65道题的MCQ-CBA形成性测评，随后完成Proxy-CBA测评；第二组（n=35）则采用相反的测试顺序。后续共有71名参与者完成了关于感知认知负荷的调查问卷。两类计算机化测评均分别基于经典测试理论（Classical Test Theory，CTT）与拉希模型（Rasch Model）的参数标准进行分析。相较于标准MCQ-CBA，Proxy-CBA的原始得分更低（p²=0.276），信度估计值更高（p²=0.498），测量标准误估计值更低（p²=0.807），且θ能力得分更低（p²=0.288）。调查问卷结果显示，两类计算机化测评在感知认知负荷方面无显著统计学差异。相较于标准MCQ-CBA，Proxy-CBA在认知负荷相当的前提下，展现出更优的信度与效度，表明其可作为替代型测评形式加以应用。