Clinical BERT Models Trained on Pseudo Re-identified MIMIC-III Notes

This project contains the weights of 7 different BERT models trained over a surrogate re-identified set of MIMIC-III (v1.4) notes. Each model is suitable for direct compatibility with the HuggingFace framework; users can easily load and apply the model. We release these model weights with the intent of facilitating research into the dangers of revealing patient information via model sharing, specifically for models pretrained on non-deidentified electronic health record (EHR) data. We additionally release our post- processed data, which most importantly contains a mapping of patient subject id to condition(s) that the given patient has. The purpose of these "subject id to condition" files is to measure how much of a correlation can be extracted from a surrogate patient name and the medical conditions that the given patient has. This will allow users to quantify the amount of Protected Health Information (PHI) "leakage" these large pretrained language models may exhibit.