BEDS-Bench

BEDS-Bench是一个用于量化机器学习模型在OOD设置下对EHR数据行为的基准。该基准使用两个开放访问的去标识化EHR数据集——MIMIC-III和PICDB，通过创建故意不相似的训练和测试集来模拟OOD设置。BEDS-Bench执行多个临床预测任务，OOD数据设置，并测量相关指标，以表征模型在OOD行为的关键方面。该基准旨在解决EHR模型在实际应用中遇到的数据分布变化时的行为和鲁棒性问题，特别是在医疗保健领域，其中模型的性能直接关系到患者的健康安全。BEDS-Bench is a benchmark designed to quantify the behavior of machine learning models on Electronic Health Records (EHR) data under out-of-distribution (OOD) settings. This benchmark employs two publicly available de-identified EHR datasets—MIMIC-III and PICDB—to simulate OOD scenarios by constructing intentionally dissimilar training and test splits. BEDS-Bench conducts multiple clinical prediction tasks across various OOD data configurations, and measures corresponding metrics to characterize critical aspects of model OOD behavior. This benchmark aims to address the behavioral and robustness issues of EHR models when encountering data distribution shifts in real-world applications, particularly in the healthcare domain where model performance is directly tied to patient health and safety.