Asclepius-R : Clinical Large Language Model Built On MIMIC-III Discharge Summaries

The development of large language models tailored for handling patients’ clinical notes is often hindered by the limited accessibility and usability of these notes due to strict privacy regulations. To address these challenges, we first create synthetic large-scale clinical notes using publicly available case reports extracted from biomedical literature. We then use these synthetic notes to train our specialized clinical large language model, Asclepius. While Asclepius is trained on synthetic data, we assess its potential performance in real-world applications by evaluating it using real clinical notes. We benchmark Asclepius against several other large language models, including GPT-3.5-turbo and other open-source alternatives. To further validate our approach using synthetic notes, we also compare Asclepius with its variants trained on real clinical notes. Our findings convincingly demonstrate that synthetic clinical notes can serve as viable substitutes for real ones when constructing high-performing clinical language models. This conclusion is supported by detailed evaluations conducted by both GPT-4 and medical professionals. All resources—including weights, codes, and data—used in the development of Asclepius are made publicly accessible for future research. Specifically, this repository contains Asclepius-R, a variant of Asclepius that was trained on MIMIC-III discharge summaries. All other resource are also publicly accessible.

针对处理患者临床记录的大语言模型之开发，往往因严格隐私法规导致的临床记录有限的可访问性和易用性而受到阻碍。为应对这些挑战，我们首先利用从生物医学文献中提取的公开可用病例报告，创建了合成的大型临床记录。随后，我们使用这些合成记录来训练我们专用的临床大语言模型——阿斯克勒庇俄斯。尽管阿斯克勒庇俄斯是在合成数据上训练的，我们仍通过使用真实临床记录对其进行评估，以评估其在现实世界应用中的潜在性能。我们将阿斯克勒庇俄斯与其他几个大型语言模型进行了基准测试，包括 GPT-3.5-turbo 以及其他开源替代方案。为进一步验证使用合成记录的方法，我们还比较了阿斯克勒庇俄斯与其在真实临床记录上训练的变体。我们的研究结果令人信服地表明，合成临床记录在构建高性能临床语言模型时，可以作为真实记录的可行替代品。这一结论得到了 GPT-4 和医疗专业人士的详细评估支持。所有用于阿斯克勒庇俄斯开发的资源，包括权重、代码和数据，均公开供未来研究使用。具体而言，本存储库包含阿斯克勒庇俄斯-R，这是一种在 MIMIC-III 出院总结上训练的阿斯克勒庇俄斯变体。所有其他资源亦均公开可访问。