Workshop on item writing to assess basic sciences in health professional education: an experience report

Abstract: Introduction: Student assessment is an essential component of all educational programs. Basic science learning is essential for making clinical knowledge meaningful to healthcare students. However, most item writer training is focused on the formulation of clinical questions and does not include a specific approach to basic science questions. Experience Report: Workshops on item writing for knowledge application on basic sciences were carried out with the aim of planning a test to be applied at the end of the basic cycle of six health courses. The instructional material was prepared by the authors, who offered online workshops. A differential of this training was the application of models of item lead-in elaboration with defined contexts, using moments of asynchronous preparation and synchronous encounter. After each workshop, surveys were applied to assess participants’ satisfaction and learning. Most participants rated the workshop as good or very good and reported an increase in their perceived ability to prepare single best answer multiple-choice questions. At the end, only 7% reported they were not prepared to write an item following good practices. There was an improvement in the quality of the items prepared, using the difficulty and discrimination indexes as a reference. Discussion: There is evidence of the value of faculty development in improving the quality of the questions produced. The proposed workshop format was well evaluated by the participants and contributed to the quality of tests applied to students at the end of the basic science cycle. Conclusion: Strategies such this qualify assessments within the school and contribute to the organization of external exams.

摘要：
引言：学生评价是所有教育项目不可或缺的核心环节。基础科学学习对于帮助医学生理解临床知识至关重要。然而，当前绝大多数试题命制培训均聚焦于临床类试题的编写，未针对基础科学类试题提供专属命制方法。
经验报告：本次研究开展了面向基础科学知识应用类试题的命制工作坊，旨在为6门健康相关课程基础学习阶段结束后的结业测试规划试题。培训材料由本文作者编写，并以线上工作坊的形式开展教学。本次培训的特色在于，结合异步准备与同步研讨环节，应用了带有明确情境的试题题干设问构建模型。每一期工作坊结束后，均通过问卷调查评估参与者的培训满意度与学习收获。绝大多数参与者将本次工作坊评为良好或优秀，并表示自身编写单项最佳答案多选题（Single Best Answer Multiple-Choice Questions）的感知能力有所提升。培训结束时，仅有7%的参与者表示尚未能按照规范流程命制试题。以难度指数与区分度指数为参考标准，参与者所命制试题的质量得到了显著提升。
讨论：现有研究证实，教师发展项目可有效提升试题命制质量。本次提出的工作坊模式获得了参与者的高度认可，且助力提升基础科学阶段结业测试的试题质量。
结论：此类优化策略可完善校内考核体系，助力校级外部考试的组织筹备工作。