Fadil369/augmented-clinical-notes

Name: Fadil369/augmented-clinical-notes
Creator: Fadil369
Published: 2026-04-05 02:23:57
License: 暂无描述

Hugging Face2026-04-05 更新2026-04-12 收录

下载链接：

https://hf-mirror.com/datasets/Fadil369/augmented-clinical-notes

下载链接

链接失效反馈

官方服务：

资源简介：

--- license: mit task_categories: - text-generation language: - en pretty_name: Augmented Clinical Notes size_categories: - 10K<n<100K configs: - config_name: default data_files: - split: train path: augmented_notes_30K.jsonl tags: - medical - health dataset_info: features: - name: idx dtype: string - name: note dtype: string - name: full_note dtype: string - name: conversation dtype: string - name: summary dtype: string --- # Augmented Clinical Notes The Augmented Clinical Notes dataset is an extension of existing datasets containing 30,000 triplets from different sources: - **Real clinical notes** (*[PMC-Patients](https://arxiv.org/abs/2202.13876)*): Clinical notes correspond to patient summaries from the PMC-Patients dataset, which are extracted from PubMed Central case studies. - **Synthetic dialogues** (*[NoteChat](https://arxiv.org/abs/2310.15959)*): Synthetic patient-doctor conversations were generated from clinical notes using GPT 3.5. - **Structured patient information** (*ours*): From clinical notes, we generate structured patient summaries using GPT-4 and a tailored medical information template (see details below). This dataset was used to train [**MediNote-7B**](https://huggingface.co/AGBonnet/medinote-7b) and [**MediNote-13B**](https://huggingface.co/AGBonnet/medinote-13b), a set of clinical note generators fine-tuned from the [**MediTron**](https://huggingface.co/epfl-llm/meditron-7b) large language models. Our full report is available [here](./report.pdf). ## Dataset Details  - **Curated by:** Antoine Bonnet and Paul Boulenger - **Language(s):** English only - **Repository:** [EPFL-IC-Make-Team/ClinicalNotes](https://github.com/EPFL-IC-Make-Team/ClinicalNotes) - **Paper:** *[MediNote: Automated Clinical Notes](report.pdf)* ## Dataset Creation **Clinical notes**. Our primary source of clinical notes is *[PMC-Patients](https://arxiv.org/abs/2202.13876)*. This large-scale dataset contains 167K patient summaries extracted from open-access case studies published in PubMed Central. Each note encapsulates a detailed case presentation as written by a doctor, presenting a thorough summary encompassing the patient’s visit, medical history, symptoms, administered treatments, as well as the discharge summary and outcome of the intervention. These comprehensive case presentations offer a rich and diverse collection of medical scenarios, forming a robust foundation for our model training and evaluation. **Synthetic dialogues**. Distribution of confidential patient-doctor conversations is forbidden, so no large scale dataset is publicly available for training. We circumvent the lack of real dialogue data by building upon [NoteChat](https://huggingface.co/datasets/akemiH/NoteChat), an extension of PMC-Patients with 167K synthetic patient-doctor conversations. Each dialogue transcript within the NoteChat dataset was generated from a clinical note by ChatGPT (version `gpt-3.5-turbo-0613`). **Patient information**. We augment the PMC-Patients and NoteChat datasets by extracting structured patient information from the 30K longest clinical notes. To do so, we prompt GPT-4 (version `gpt-4-turbo-0613`) with zero-shot instructions, providing clinical notes and a structured template of patient medical information with feature definitions. This template, shown below, encapsulates crucial aspects of a clinical note such as the patient’s admission to a care center, medical history, current symptoms, as well as the doctor’s diagnosis and treatment plan. The full data pipeline is shown below. <img width=70% src="data_pipeline.pdf" alt="Data pipeline" title="Data pipeline"> ### Medical information template Here is shown the medical template we used to structurize clinical notes. A JSON version is also available as `template_definitions.json`. <img width=70% src="template.pdf" alt="Data pipeline" title="Data pipeline"> ### Dialogue Quality The primary aim of synthetic dialogues is to distill comprehensive information from the case presentation, transforming it into a plausible and engaging conversation. Newer versions of the dataset include higher quality dialogues generated by GPT-4 and NoteChat, a multi-agent dialogue generation pipeline (see the [NoteChat repository](https://github.com/believewhat/Dr.NoteAid) for more information). Dialogues produced by ChatGPT tend to lack realism and frequently adhere to a pattern where the doctor poses a series of questions mirroring the facts from the original clinical notes, receiving simple ’Yes’ responses from the patient. Nevertheless, we decided to use ChatGPT dialogues as they were the only ones available during the training phase. Clinical notes within NoteChat were truncated prior to the dialogue generation process. Consequently, the information lost due to truncation from the clinical note is also missing in the resulting dialogue. While complete notes were accessible from PMC-Patients, a conscious decision was made to fine-tune our models using truncated notes. This decision aimed at preventing our fine-tuned models from being inadvertently trained to hallucinate information towards the conclusion of a note. Notably, certain ChatGPT dialogues involving scenarios where a patient passes away and a subsequent dialogue with a family member commences revealed instances of prompt leaks. These leaks manifested as the prompt used for synthetic dialogue generation being inadvertently repeated within the dialogue. ## Dataset Structure  Each row of the dataset represents one dialogue-summary-note triplet, and consists of the following dataset fields (all strings): | Field | Description | Source | |-|-|-| | `idx` | Unique identifier, index in the original NoteChat-ChatGPT dataset | NoteChat | | `note` | Clinical note used by NoteChat (possibly truncated) | NoteChat | | `full_note` | Full clinical note | PMC-Patients | | `conversation` | Patient-doctor dialogue | NoteChat | | `summary`| Patient information summary (JSON) | ours | ## Uses  While this dataset was originally used to fine-tune LLMs to extract structured patient information from dialogue, it can also be used for diverse applications in the healthcare domain, such as training models to extract comprehensive tabular patient features from clinical notes. ## Bias, Risks, and Limitations  - **Synthetic Data**: NoteChat dialogues were synthetically generated from clinical notes; they are not completely realistic and therefore fail to accurately represent real patient-doctor conversations. Real patient-doctor conversations are of course preferred, but their distribution is forbidden in the US by the [Health Insurance Portability and Accountability Act of 1996](https://www.cdc.gov/phlp/publications/topic/hipaa.html). - **Representation**: PMC-Patients clinical notes have been extracted from English PubMed Central publications, and therefore over-represent clinical settings from English-speaking countries. ## Acknowledgments We thank Prof. Mary-Anne Hartley for her advice on the appropriate template for structured medical patient summaries. <!-- ## Citation If you use the Augmented Clinical Notes dataset, please cite out work: ``` ADD CITATION ``` --!>

license: MIT协议 task_categories: - 文本生成 language: - 英语 pretty_name: 增强型临床笔记 size_categories: - 10000 < 样本数 < 100000 configs: - config_name: 默认配置 data_files: - split: 训练集 path: augmented_notes_30K.jsonl tags: - 医疗 - 健康 dataset_info: features: - name: idx dtype: 字符串 - name: note dtype: 字符串 - name: full_note dtype: 字符串 - name: conversation dtype: 字符串 - name: summary dtype: 字符串 # 增强型临床笔记本增强型临床笔记数据集是现有数据集的扩展，包含来自三类不同来源的30000组三元组： - **真实临床笔记**（*[PMC-Patients]("https://arxiv.org/abs/2202.13876")*）：对应PMC-Patients数据集中的患者摘要，这些摘取自PubMed Central的病例研究。 - **合成对话**（*[NoteChat]("https://arxiv.org/abs/2310.15959")*）：使用GPT-3.5基于临床笔记生成的模拟医患对话。 - **结构化患者信息**（*本团队自制*）：从临床笔记中，通过GPT-4与定制化医疗信息模板（详见下文）生成结构化患者摘要。本数据集曾用于训练[**MediNote-7B**]("https://huggingface.co/AGBonnet/medinote-7b")与[**MediNote-13B**]("https://huggingface.co/AGBonnet/medinote-13b")，这两款临床笔记生成模型均基于[**MediTron**]("https://huggingface.co/epfl-llm/meditron-7b")大语言模型（Large Language Model）进行微调。我们的完整报告可通过[此处](./report.pdf)获取。 ## 数据集详情 - **整理方**：Antoine Bonnet与Paul Boulenger - **语言**：仅英语 - **代码仓库**：[EPFL-IC-Make-Team/ClinicalNotes]("https://github.com/EPFL-IC-Make-Team/ClinicalNotes") - **相关论文**：*[MediNote: Automated Clinical Notes](report.pdf)* ## 数据集构建 ### 临床笔记来源本数据集的临床笔记主要源自[PMC-Patients]("https://arxiv.org/abs/2202.13876")。该大规模数据集包含167000条摘取自PubMed Central开源病例研究的患者摘要。每条笔记均由医生撰写，完整呈现了患者的就诊记录、病史、症状、所接受的治疗方案，以及出院小结与干预结局等详细病例信息。这些全面的病例展示涵盖了丰富多样的医疗场景，为模型训练与评估提供了坚实的基础。 ### 合成对话构建由于医患对话涉及隐私，其公开分发在美国受《1996年健康保险流通与责任法案》（HIPAA）限制，目前尚无大规模公开的真实医患对话数据集可供训练。为此，我们基于NoteChat数据集进行拓展——NoteChat是PMC-Patients的扩展数据集，包含167000条模拟医患对话。NoteChat中的每条对话均由ChatGPT（版本`gpt-3.5-turbo-0613`）基于临床笔记生成。 ### 患者信息结构化处理我们对PMC-Patients与NoteChat数据集进行了增强，从30000条最长的临床笔记中提取结构化患者信息。具体流程为：使用**零样本（Zero-shot）**提示向GPT-4（版本`gpt-4-turbo-0613`）发送请求，提供临床笔记与包含特征定义的结构化患者医疗信息模板。该模板涵盖了临床笔记的核心要素，如患者就诊信息、病史、当前症状、医生诊断与治疗方案等。完整的数据处理流程如下图所示。 <img width=70% src="data_pipeline.pdf" alt="数据处理流程" title="数据处理流程"> ### 医疗信息模板以下为我们用于结构化临床笔记的医疗模板，其JSON版本可通过`template_definitions.json`获取。 <img width=70% src="template.pdf" alt="医疗信息模板" title="医疗信息模板"> ### 对话质量合成对话的核心目标是从病例展示中提炼全面信息，并转化为逼真且自然的对话。本数据集的新版本包含由GPT-4与NoteChat多智能体对话生成管道生成的更高质量对话（详见[NoteChat代码仓库]("https://github.com/believewhat/Dr.NoteAid")）。 ChatGPT生成的对话往往缺乏真实感，常遵循固定模式：医生按照原始临床笔记中的事实依次提问，患者仅以简单的“是”作答。但考虑到训练阶段仅能获取此类对话数据，我们最终仍选择使用ChatGPT生成的对话。 NoteChat中的临床笔记在对话生成前已被截断，因此因截断丢失的信息也不会出现在最终的对话中。尽管PMC-Patients提供了完整的临床笔记，但我们仍选择使用截断后的笔记对模型进行微调，以避免模型在训练过程中无意识地学习到在笔记末尾编造信息的行为。值得注意的是，部分涉及患者死亡及后续与家属对话的ChatGPT对话中出现了提示词泄露问题：用于生成合成对话的提示词被意外重复嵌入对话内容中。 ## 数据集结构本数据集的每一行代表一组对话-摘要-笔记三元组，包含以下全部为字符串类型的字段： | 字段 | 描述 | 来源 | |-|-|-| | `idx` | 唯一标识符，对应原始NoteChat-ChatGPT数据集中的索引 | NoteChat | | `note` | NoteChat使用的临床笔记（可能已被截断） | NoteChat | | `full_note` | 完整临床笔记 | PMC-Patients | | `conversation` | 医患对话 | NoteChat | | `summary` | 患者信息摘要（JSON格式） | 本团队自制 | ## 数据集用途尽管本数据集最初用于微调大语言模型以从对话中提取结构化患者信息，但其也可应用于医疗领域的多种场景，例如训练模型从临床笔记中提取全面的结构化患者特征。 ## 偏差、风险与局限性 - **合成数据局限性**：NoteChat中的对话均由临床笔记合成生成，并非完全真实的医患对话，因此无法准确还原真实的医患交流场景。尽管真实医患对话是更优选择，但根据美国《1996年健康保险流通与责任法案》（HIPAA），其公开分发受到严格限制。 - **代表性偏差**：PMC-Patients的临床笔记摘取自英文PubMed Central出版物，因此该数据集过度代表了英语国家的临床场景。 ## 致谢我们感谢Mary-Anne Hartley教授为结构化医疗患者摘要模板提供的专业建议。

提供机构：

Fadil369