Data from: Comparison of the validity of the checklist assessment in the cardiac arrest simulations with the app in an academic hospital in Taiwan: a retrospective observational study

Background: Robust assessment is a crucial component in Advanced Cardiac
Life Support (ACLS) training to determine whether participants have
achieved learning objectives with little or no variation in their overall
outcomes. This study aimed to evaluate resuscitation performance by
real-time logs. We hypothesized that instructors may not be able to
evaluate time-sensitive parameters, namely, chest compression fraction,
time to initiating chest compression, and time to initiating
defibrillation efficiently in a subjective manner. Methods: Video records
and formal checklist-based test results of Megacode scenarios for the ACLS
certification examination at several hospitals in Taipei were examined.
For the study interest, three time-sensitive parameters were measured via
video review assisted by a mobile phone application, and were used for
evaluation. We evaluated if the pass/fail results made by instructors via
checklists were correlated with these parameters. Results: A total of 185
Megacode scenarios were eligible for the final analysis. Among the three
parameters, good chest compression fraction was statistically significant
with higher odds ratio (OR) of passing (OR = 3.65; 95% confidence interval
[CI]: 1.36-9.91; P = 0.01). In 112 participants with one parameter that
did not meet the criteria, 25 were graded as fail, making the specificity
22.3% (95% CI: 15.0-31.2%). Conclusions: Visual observation of
cardiopulmonary resuscitation performance is not accurate when evaluating
time-sensitive parameters. Objective results should be offered for
training outcome evaluation, and also for feedback to participants.