Unequal discrimination at comparable difficulty: a field comparison of cognitive-model-guided LLM-generated and faculty-curated multiple-choice items across three summative examinations
收藏资源简介:
De-identified item-level response data, generation artefacts and analysis scripts for a field comparison of 58 multiple-choice questions generated by a cognitive-model-guided GPT-4o pipeline and 43 items from a curated faculty item bank, administered interleaved in three consecutive summative sixth-year Medicine examinations (Neurology, Endocrinology, Hematology; University of Kerbala, August 2026; 90 examinees each). The deposit contains: the raw optical-mark-recognition response exports with student names replaced by anonymous IDs; source maps giving each question's source and corrected key; the generation artefact (cognitive model, vignette, options, key and rationale) for all 60 generated pipeline items, 58 of them administered; the three examination blueprints; per-item statistics; item-level labels from a blinded review of option-set quality covering all 102 administered items; the source code of the item-generation pipeline; and Python scripts that reproduce every result in the accompanying manuscript. Faculty item text is not included, and the text of textbook passages the pipeline retrieved has been removed for copyright reasons (sources and ranks are kept). The README documents which fields of each generation artefact were seen by candidates, the post-examination key corrections and removals from scoring, and one known content defect in a deposited item, which is reported rather than edited out.



