Code for: Comparison and interpretability of machine learning models to predict severity of chest injury

Objective: Trauma quality improvement programs and registries
improve care and outcomes for injured patients. Designated trauma centers
calculate injury scores using dedicated trauma registrars; however, many
injuries arrive at non-trauma centers, leaving a substantial amount of
data uncaptured. We propose automated methods to identify severe chest
injury using machine learning (ML) and natural language processing (NLP)
methods from the electronic health record (EHR) for quality reporting.
Materials and Methods: A level I trauma center was queried for
patients presenting after injury between 2014 and 2018. Prediction
modeling was performed to classify severe chest injury using a reference
dataset labeled by certified registrars. Clinical documents from trauma
encounters were processed into concept unique identifiers for inputs to ML
models: logistic regression with elastic net regularization (EN), extreme
gradient boosted machines (XGB), and convolutional neural networks (CNN).
The optimal model was identified by examining predictive and face validity
metrics using global explanations. Results: Of 8,952 encounters,
542 (6.1%) had a severe chest injury. CNN and EN had the highest
discrimination, with an area under the receiver operating characteristic
curve of 0.93 and calibration slopes between 0.88 and 0.97. CNN had better
performance across risk thresholds with fewer discordant cases.
Examination of global explanations demonstrated the CNN model had better
face validity, with top features including “contusion of lung” and
“hemopneumothorax.”  Discussion: The CNN model featured
optimal discrimination, calibration, and clinically relevant features
selected.  Conclusion: NLP and ML methods to populate
trauma registries for quality analyses are feasible.