Clinical BERT Models Trained on Pseudo Re-identified MIMIC-III Notes

This project contains the weights of 7 different BERT models trained over a
surrogate re-identified set of MIMIC-III (v1.4) notes. Each model is suitable
for direct compatibility with the HuggingFace framework; users can easily load
and apply the model. We release these model weights with the intent of
facilitating research into the dangers of revealing patient information via
model sharing, specifically for models pretrained on non-deidentified
electronic health record (EHR) data. We additionally release our post-
processed data, which most importantly contains a mapping of patient subject
id to condition(s) that the given patient has. The purpose of these "subject
id to condition" files is to measure how much of a correlation can be
extracted from a surrogate patient name and the medical conditions that the
given patient has. This will allow users to quantify the amount of Protected
Health Information (PHI) "leakage" these large pretrained language models may
exhibit.