ClinicNet: machine learning for personalized order set recommendations

Objective

This study assesses whether neural networks trained on electronic health record (EHR) data can anticipate what individual clinical orders and existing institutional order set templates clinicians will use more accurately than existing decision support tools.

Materials and Methods

We process 57,624 patients-worth of clinical event EHR data from 2008-2014. We train a feed-forward neural network (ClinicNet) and logistic regression applied to the traditional problem structure of predicting individual clinical items  as well as our proposed workflow of predicting existing institutional order set template usage.

Results

ClinicNet predicts individual clinical orders (precision=0.32, recall=0.47) better than existing institutional order sets (precision=0.15, recall=0.46). The ClinicNet model predicts clinician usage of existing institutional order sets (avg. precision=0.31) with higher average precision than  a baseline of order set usage frequencies (avg. precision=0.20) or a logistic regression model (avg. precision=0.12).

Discussion

Machine learning methods can predict clinical decision-making patterns with greater accuracy and less manual effort than existing static order set templates. This can streamline existing clinical workflows, but may not fit if historical clinical ordering practices are incorrect. For this reason, manually authored content such as order set templates remain valuable for purposeful design of care pathways. ClinicNet’s capability of predicting such personalized order set templates illustrates the potential of combining both top-down and bottom-up approaches to delivering clinical decision support content.

Conclusion

ClinicNet illustrates the capability for machine learning methods applied to the EHR to anticipate both individual clinical orders and existing order set templates, which has the potential to improve upon current standards of practice in clinical order entry.

Methods
See manuscript Methods section.