bio-nlp-umass/Synth-SBDH

--- license: apache-2.0 task_categories: - text-classification - token-classification language: - en tags: - me - croissant pretty_name: Synth-SBDH size_categories: - 1K<n<10K --- # Dataset Card for Synth-SBDH Synth-SBDH is a collection of 8,767 synthetic examples with annotations for 15 SBDH categories. SBDH annotations include information such as presence, period and annotation rationale. ## Dataset Description Synth-SBDH is a novel synthetic SBDH dataset that mimics EHR notes. - **Repository:** [Codes to reproduce experiments](https://github.com/avipartho/Synth-SBDH) - **Paper:** [Link](https://aclanthology.org/2025.emnlp-main.1418/) - **Point of Contact:** [Avijit Mitra](mailto:[e-mail]) ## Dataset Structure ### Data Instances Some examples from [synth_sbdh_train.csv](synth_sbdh_train.csv) looks as follows. ``` { 'ex_no': 30, 'Text': 'Patient has lost his job due to physical disabilities and is currently living on government financial aid.', 'Textspan': 'lost his job || living on government financial aid', 'SBDH': 'job insecurity || financial insecurity', 'Presence': 'yes || yes', 'Period': 'current || current', 'Reasoning': 'The patient lost his job due to physical issues and this refers to job insecurity. || Reliance on government financial aid signifies financial insecurity.' } { 'ex_no': 31, 'Text': 'Patient was assaulted last year and suffers from PTSD.', 'Textspan': 'assaulted || suffers from PTSD', 'SBDH': 'violence || psychiatric symptoms or disorders', 'Presence': 'yes || yes', 'Period': 'history || current', 'Reasoning': 'Being assaulted is a form of violence. || PTSD is a psychiatric disorder.' } ``` ### Data Fields We release Synth-SBDH as CSV files. Each CSV file has the following fields: - `ex_no`: Unique identifier for an example. - `Text`: Example text sequence, at max a few sentences long. - `Textspan`: Text spans with mentions of SBDH, separated by '||'. - `Reasoning`: Rationales for SBDH annotations, separated by '||'. - `SBDH`: SBDH annotations for text spans in `Textspan`, separated by '||'. - `Presence`: Presence annotations (yes/no) for text spans in `Textspan`, separated by '||'. - `Period`: Period annotations (current/history) for text spans in `Textspan`, separated by '||'. - `Operation` (only for [synth_sbdh_test_reviewed.csv](synth_sbdh_test_reviewed.csv) file): One of the four review operation considered by the human experts - *keep*, *correct*, *discard* or *add*. ### Data Splits The Synth-SBDH dataset has 4 splits: _train_, _val_, _test_, and _test_reviewed_. Below are the statistics for the dataset. | Dataset Split | Number of Examples | Number of Annotations | | ------------- | ------------------ | --------------------- | | Train | 6,136 | 10,022 | | Val | 876 | 1,443 | | Test | 1,755 | 2,904 | | Test (Expert Reviewed) | 1,732 | 3,345 | ## Dataset Creation Details about how the data was created are available in our paper. ## Uses You may directly download and use the dataset using the `datasets` library. ```python from datasets import load_dataset synth_sbdh_dataset = load_dataset("bio-nlp-umass/Synth-SBDH") ``` Or you can also individually download the files and load them using any compatible libray. For example, using `pandas` - ```python import pandas as pd synth_sbdh_df = pd.read_csv('FILE_NAME') ``` ## Citation ``` @inproceedings{mitra-etal-2025-synth, title = "Synth-{SBDH}: A Synthetic Dataset of Social and Behavioral Determinants of Health for Clinical Text", author = "Mitra, Avijit and Yang, Zhichao and Druhl, Emily and Goodwin, Raelene and Yu, Hong", editor = "Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet", booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing", month = nov, year = "2025", address = "Suzhou, China", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.emnlp-main.1418/", doi = "10.18653/v1/2025.emnlp-main.1418", pages = "27899--27935", ISBN = "979-8-89176-332-6", abstract = "Social and behavioral determinants of health (SBDH) play a crucial role in health outcomes and are frequently documented in clinical text. Automatically extracting SBDH information from clinical text relies on publicly available good-quality datasets. However, existing SBDH datasets exhibit substantial limitations in their availability and coverage. In this study, we introduce Synth-SBDH, a novel synthetic dataset with detailed SBDH annotations, encompassing status, temporal information, and rationale across 15 SBDH categories. We showcase the utility of Synth-SBDH on three tasks using real-world clinical datasets from two distinct hospital settings, highlighting its versatility, generalizability, and distillation capabilities. Models trained on Synth-SBDH consistently outperform counterparts with no Synth-SBDH training, achieving up to 63.75{\%} macro-F improvements. Additionally, Synth-SBDH proves effective for rare SBDH categories and under-resource constraints while being substantially cheaper than expert-annotated real-world data. Human evaluation reveals a 71.06{\%} Human-LLM alignment and uncovers areas for future refinements." } ```Synth-SBDH is a collection of 8,767 synthetic examples with annotations for 15 SBDH categories. SBDH annotations include information such as presence, period and annotation rationale. It is a novel synthetic SBDH dataset that mimics EHR notes.

搜集汇总

数据集介绍

构建方式

在临床自然语言处理中，健康的社会与行为决定因素（SBDH）自动化抽取依赖兼具隐私安全与标注质量的语料，Synth-SBDH即为此类合成资源。依据数据集卡片与论文说明，该数据集通过合成方式模拟电子健康记录（EHR）笔记的文本形态，围绕15类SBDH生成8,767个实例，并为文本片段提供存在性、时间属性与标注理由等细粒度信息。数据划分为训练、验证、测试及专家复核测试四个部分，其中专家复核子集记录keep、correct、discard或add等审查操作。构建流程与生成细节详见原论文，代码库亦提供复现实验所需材料。

特点

面向临床文本中SBDH信息抽取，语料需同时覆盖类别广度、时间维度与可解释依据，Synth-SBDH由此展现出合成数据特有的设计取向。其涵盖15个SBDH类别，标注不仅包含文本片段与类别，还记录存在与否、当前或历史时段以及推理理由，因而可支撑文本分类与token分类等任务。数据集总计8,767例，训练、验证、测试与专家复核测试分别具有明确规模；专家复核部分附带操作标签，有助于分析标注修正与增补。论文报告显示，基于该数据训练的模型在真实临床数据上可取得显著macro-F提升，并对稀有类别与低资源场景具有助益，人机对齐程度亦经人工评估。

使用方法

在SBDH模型开发与评测实践中，研究者可获取并利用Synth-SBDH开展训练、微调与迁移评估。数据集可通过Hugging Face datasets库以bio-nlp-umass/Synth-SBDH标识直接加载，亦可分别下载CSV文件并使用pandas等兼容工具读取。每条记录包含ex_no、Text、Textspan、SBDH、Presence、Period与Reasoning等字段，测试复核文件另含Operation字段，用于表征专家审查操作。使用者可据任务需求将文本片段与类别映射为分类或序列标注目标，并结合各划分进行模型比较、错误分析与跨机构泛化检验。

背景与挑战

背景概述

Synth-SBDH 由马萨诸塞大学阿默斯特分校 bio-nlp-umass 研究团队于 2025 年提出，相关论文发表于 EMNLP 2025，作者包括 Avijit Mitra、Zhichao Yang、Emily Druhl、Raelene Goodwin 与 Hong Yu。该数据集面向临床文本中社会与行为健康决定因素（SBDH）自动抽取，包含 8,767 条合成样本，覆盖 15 个 SBDH 类别，并标注存在性、时期与标注理据。其核心研究问题在于缓解现有 SBDH 资源可用性不足、类别覆盖有限及真实临床数据标注昂贵等困境，通过在两家医院真实数据上的三项任务验证，训练于 Synth-SBDH 的模型 macro-F 最高提升 63.75%，并展现稀有类别适应与知识蒸馏潜力。

当前挑战

Synth-SBDH 所应对的领域问题在于从非结构化电子健康记录中精准识别社会与行为健康决定因素，其难点涵盖多标签语义交织、时间状态判别、类别长尾分布、隐私约束下的真实标注稀缺，以及跨机构临床语境的泛化衰减。构建层面，合成语料需在语言风格与语义分布上逼近真实 EHR 笔记，同时为 15 类 SBDH 提供存在性、当前或历史时期及推理依据，保证跨字段一致性；专家审查 test_reviewed 还需以 keep、correct、discard、add 四类操作修正模型生成结果，使合成数据兼具规模、质量与临床可信度。Human-LLM 对齐率为 71.06%，亦揭示合成标注与人类判断之间仍存改进空间。

常用场景

经典使用场景

在临床自然语言处理领域，社会与行为健康决定因素（SBDH）的自动识别长期依赖高质量标注语料。Synth-SBDH 最经典的使用场景是面向电子健康记录文本的 SBDH 抽取与分类，涵盖15类社会行为风险，如职业不稳定、财务不安全、暴力暴露与精神症状等。研究者可将其用于文本分类与序列标注任务，训练模型识别文本片段中的 SBDH 提及，并同步判断其存在状态、当前或历史时段及标注理由。凭借合成样本对真实病历笔记的模拟，该数据集为 SBDH 识别模型提供了可复现、低成本且覆盖较广的预训练与评测基础。

解决学术问题

围绕 SBDH 计算研究，现有公开数据集常受制于隐私壁垒、标注成本高昂、类别覆盖稀疏以及时间信息与解释依据缺失等局限，导致模型泛化评估困难，罕见社会风险识别效果欠佳。Synth-SBDH 以合成方式构建大规模标注实例，系统性覆盖15类 SBDH，并保留存在性、时段和标注理由，为临床文本分类与序列标注提供可共享资源。其学术意义在于缓解真实临床数据不可获取与标注昂贵之间的矛盾，支持跨机构泛化、低资源学习和模型蒸馏研究，推动 SBDH 抽取从少量类别向更全面、更具时间敏感性的方向演进，并为健康公平相关计算研究提供可检验基准。

衍生相关工作

Synth-SBDH 的发布催生了一系列围绕临床 SBDH 抽取的后续研究。相关经典工作多聚焦于利用该合成语料进行跨机构迁移、知识蒸馏与低资源适配，验证其在真实医院数据上的泛化潜力；亦有研究以它为基准，比较大语言模型与监督模型在 SBDH 分类、时段判定和理由生成中的表现，并开展人类与模型标注一致性评估。部分工作进一步探索罕见 SBDH 类别的识别策略、文本片段级序列标注方法，以及合成数据与专家标注数据混合训练的效果。这些衍生研究共同拓展了 SBDH 计算的方法边界，并为临床 NLP 社区提供了可复用的评测范式与改进方向。