Development and Simulation Testing of Philadelphia Naming Test (Hula et al., 2015)

Purpose The purpose of this study was to develop a computerized adaptive test (CAT) version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996), to reduce test length while maximizing measurement precision. This article is a direct extension of a companion article (Fergadiotis, Kellough, & Hula, 2015), in which we fitted the PNT to a 1-parameter logistic item-response-theory model and examined the validity and precision of the resulting item parameter and ability score estimates. Method Using archival data collected from participants with aphasia, we simulated two PNT-CAT versions and two previously published static PNT short forms, and compared the resulting ability score estimates to estimates obtained from the full 175-item PNT. We used a jackknife procedure to maintain independence of the samples used for item estimation and CAT simulation. Results The PNT-CAT recovered full PNT scores with equal or better accuracy than the static short forms. Measurement precision was also greater for the PNT-CAT than the static short forms, though comparison of adaptive and static nonoverlapping alternate forms showed minimal differences between the two approaches. Conclusion These results suggest that CAT assessment of naming in aphasia has the potential to reduce test burden while maximizing the accuracy and precision of score estimates. In this article, we extend work presented in a companion article (Fergadiotis, Kellough, & Hula, 2015) to construct and evaluate an item response theory (IRT)–based computer adaptive version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996). Using simulations based on responses previously collected from participants with aphasia, we evaluated agreement between computer adaptive short forms and the full PNT and compared the results to those obtained using recently developed static short forms (Walker & Schwartz, 2012). We also evaluated the equivalence of alternate test forms created by the adaptive-testing algorithm.

研究目的：本研究旨在开发费城命名测验（Philadelphia Naming Test, PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996）的计算机化自适应测验（computerized adaptive test, CAT）版本，以期在缩短测验时长的同时最大化测量精度。本文为配套研究（Fergadiotis, Kellough, & Hula, 2015）的直接延伸，该配套研究将PNT拟合至单参数logistic项目反应理论模型（1-parameter logistic item-response-theory model），并检验了所得项目参数与能力得分估计值的效度与精度。

研究方法：本研究利用从失语症（aphasia）受试者处采集的存档数据，模拟构建了两种PNT计算机化自适应测验版本与两种已发表的静态PNT简短版测验（static short forms），并将所得能力得分估计值与完整175题版PNT的估计值进行对比。本研究采用刀切法（jackknife procedure）以确保项目估计与自适应测验模拟所用样本的独立性。

研究结果：相较于静态简短版测验，PNT计算机化自适应测验能够以同等或更高精度还原完整PNT得分。其测量精度亦优于静态简短版测验，但对自适应与静态非重叠替代测验形式的对比分析显示，两种方法间差异极小。

研究结论：上述结果表明，针对失语症命名能力的计算机化自适应测验评估，有望在减轻测验负担的同时最大化得分估计的准确性与精度。本文进一步拓展了配套研究（Fergadiotis, Kellough, & Hula, 2015）的工作，构建并评估了基于项目反应理论（item response theory, IRT）的费城命名测验（PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996）计算机化自适应版本。本研究利用此前从失语症受试者处收集的作答数据开展模拟，评估了计算机化自适应简短版测验与完整PNT间的一致性，并将结果与新近开发的静态简短版测验（Walker & Schwartz, 2012）所得结果进行对比。此外，本研究还评估了自适应测验算法生成的替代测验形式的等效性。