Assessment tool committee demographics.

IntroductionValid and reliable scores from measurement tools to test competency in basic manual wheelchair-service-provision are needed to promote good practice and support capacity building. The International Society of Wheelchair Professionals’ (ISWP) Basic Test Version 1 in English, launched in 2015, is the most frequently used outcome measure tool to test basic manual wheelchair-service-provision knowledge and is part of an international certification process. Despite the wide acceptance and use of the test, its psychometric properties have not yet been established. The objectives of this study were 1) to evaluate the test’s psychometric properties, 2) to develop the test’s Version 2, and 3) to evaluate the content validity of the new version.MethodsFor Objective 1, methods from the Classical Test Theory were used to obtain items’ difficulty, item discrimination index and domains’ reliability. For Objective 2, a team of experts in wheelchair service delivery and education conducted a systematic qualitative review of the questions’ text and answers and updated them using evidence-based guidelines. For Objective 3, an external team reviewed the clarity, relevance and domain allocation of the developed items using a 4-point Likert scale. Descriptive statistics were used to describe and characterize the results for each objective. Item-content (I-CVI) and Scale-content (S-CVI) validity indexes were calculated to compute content validity.ResultsFor Objective 1, all domains in the test were below the threshold for acceptable internal consistency reliability; 80% of the total test pool (116 items from the total pool of 145) did not meet the thresholds for item difficulty and index of discrimination suggested in the literature. Of the items in the Test, 78% could be responded to intuitively and 66% did not distinguish between test-takers who were knowledgeable in the content area and those who were not. For Objective 2, experts found concerns such as items being grouped in the wrong domain, being repeated, not using person-first language, and using terms inconsistently. Thirty-four (23.4%) items were dropped and 111 (76.5%) were updated. In addition, 61 new items were developed. Members re-categorized the items and proposed a new classification of subdomains. For Objective 3, good agreement between subject-matter experts was found; the S-CVI calculated using the I-CVIs related to item clarity was 84% while using the I-CVIs related to item relevance was 98%. Only 7 items (4.1%) were deemed to be in the wrong domain and 4 items (2.3%) were considered irrelevant and dropped.ConclusionThe psychometric evidence in support of ISWP Basic Test Version 1 in English is suboptimal. A new set of items developed by experts in the field has shown excellent content validity. Ongoing assessments will be needed as ISWP Basic Test Version 2 is implemented and monitored.

引言：当前为测评基础手动轮椅服务胜任力，亟需具备良好信效度的测评工具，以推动行业规范实践并支持能力建设工作。2015年推出的国际轮椅专业人士协会（International Society of Wheelchair Professionals, ISWP）英文基础测试版本1，是目前应用最广泛的基础手动轮椅服务知识测评工具，同时也是国际认证流程的组成部分。尽管该测试已获得广泛认可与应用，但其心理测量学属性尚未得到明确验证。本研究的目标共有三项：1）评估该测试的心理测量学属性；2）开发该测试的版本2；3）对新版本的内容效度进行测评。

方法：针对目标1，本研究采用经典测验理论（Classical Test Theory）的方法，计算试题难度、试题区分度指标以及各维度的信度。针对目标2，由轮椅服务交付与教育领域的专家组成团队，对试题文本与答案进行系统性质性审查，并基于循证指南对其进行修订。针对目标3，由外部团队采用4级李克特（Likert）量表，对开发完成的试题的清晰度、相关性以及维度归属进行评审。本研究采用描述性统计方法对各目标的结果进行描述与特征分析，并通过计算试题内容效度指数（Item-content Validity Index, I-CVI）与量表内容效度指数（Scale-content Validity Index, S-CVI）来评估内容效度。

结果：针对目标1，该测试的所有维度内部一致性信度均未达到可接受阈值；总试题库（共145道试题）中的80%（116道试题）未达到文献中提出的试题难度与区分度指标阈值。在该测试的试题中，78%的试题可凭直觉作答，66%的试题无法有效区分具备该领域知识的应试者与不具备该领域知识的应试者。针对目标2，专家团队发现了诸多问题：试题被错误归类至不当维度、试题重复、未使用以人为先的语言表述，以及术语使用不统一等。本次修订共删除34道试题（占比23.4%），修订111道试题（占比76.5%），此外新增61道试题。团队成员对试题进行了重新分类，并提出了子维度的全新分类框架。针对目标3，领域专家达成了良好的一致性：基于试题清晰度的I-CVI计算得到的S-CVI为84%，基于试题相关性的I-CVI计算得到的S-CVI为98%。仅有7道试题（占比4.1%）被判定为维度归属错误，4道试题（占比2.3%）被认定为不相关并予以删除。

结论：现有证据表明，英文版本ISWP基础测试版本1的心理测量学属性未达理想标准。本领域专家开发的全新试题库展现出优异的内容效度。随着ISWP基础测试版本2的推行与监管，仍需开展持续性的评估工作。