Medical interview score data from PostCC-OSCE and programs for an extended many-facet IRT model

Objective structured clinical examinations (OSCEs) are widely used
performance assessments for medical and dental students. A common
limitation of OSCEs is that the evaluation results depend on the
characteristics of raters and the scoring rubric. To overcome this
limitation, item response theory (IRT) models such as the many-facet
models have been proposed to estimate examinee abilities while accounting
for the characteristics of raters and evaluation items in a rubric.
However, conventional IRT models have two impractical assumptions:
constant rater severity across all evaluation items in a rubric and an
equal interval rating scale among evaluation items, which can decrease
model fitting and ability measurement accuracy. To resolve this problem,
we propose a new IRT model that relaxes these assumptions. We demonstrate
the effectiveness of the proposed model by applying it to actual data
collected from a medical interview test conducted at Tokyo Medical and
Dental University as part of a post-clinical clerkship (PostCC) OSCE. The
experimental results showed that the proposed model fit our OSCE data well
and measured ability accurately. Furthermore, it provided abundant
information on rater and item characteristics that conventional models
cannot, helping us to better understand rater and item properties. This
dataset includes the actual score data collected from the above-mentioned
medical interview test in a PostCC OSCE, as well as the program for
estimating the parameters of the proposed IRT model.