MichiganNLP/HeadRoom
收藏资源简介：
--- license: mit task_categories: - text-classification - zero-shot-classification - text-generation language: - en tags: - medical - LLM - depression - race - gender pretty_name: HeadRoom size_categories: - 1K<n<10K --- # Dataset Card for InspAIred ## Table of Contents - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Languages](#languages) - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards) - [Additional Information](#additional-information) - [Citation Information](#citation-information) - [Contributions](#contributions) ## Dataset Description - **Homepage:** [HeadRoom homepage](https://github.com/MichiganNLP/depression_synthetic_data) - **Repository:** [HeadRoom repository](https://github.com/MichiganNLP/depression_synthetic_data) - **Paper:** [Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data ](https://arxiv.org/abs/2403.16909) - **Point of Contact:** [Shinka Mori](mailto:shinkamo@umich.edu ) ### Dataset Summary This work proposes to study the application of GPT-3 as a synthetic data generation tool for mental health, by analyzing its Algorithmic Fidelity, a term coined by Argyle et al 2022 to refer to the ability of LLMs to approximate real-life text distributions. Using GPT-3, we develop HeadRoom, a synthetic dataset of 3,120 posts about depression-triggering stressors, by controlling for race, gender, and time frame (before and after COVID-19). We hope our work contributes to the study of synthetic data generation and helps researchers analyze and understand how closely GPT-3 can mimic real-life depression data. ### Languages The text in the dataset is in English. ### Supported Tasks and Leaderboards TODO ## Additional Information ### Citation Information ```bibtex @inproceedings{mori-etal-2024-towards-algorithmic, title = "Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data", author = "Mori, Shinka and Ignat, Oana and Lee, Andrew and Mihalcea, Rada", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)", month = may, year = "2024", address = "Torino, Italy", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.1423", pages = "16378--16391", abstract = "Synthetic data generation has the potential to impact applications and domains with scarce data. However, before such data is used for sensitive tasks such as mental health, we need an understanding of how different demographics are represented in it. In our paper, we analyze the potential of producing synthetic data using GPT-3 by exploring the various stressors it attributes to different race and gender combinations, to provide insight for future researchers looking into using LLMs for data generation. Using GPT-3, we develop HeadRoom, a synthetic dataset of 3,120 posts about depression-triggering stressors, by controlling for race, gender, and time frame (before and after COVID-19). Using this dataset, we conduct semantic and lexical analyses to (1) identify the predominant stressors for each demographic group; and (2) compare our synthetic data to a human-generated dataset. We present the procedures to generate queries to develop depression data using GPT-3, and conduct analyzes to uncover the types of stressors it assigns to demographic groups, which could be used to test the limitations of LLMs for synthetic data generation for depression data. Our findings show that synthetic data mimics some of the human-generated data distribution for the predominant depression stressors across diverse demographics.", } ``` ### Contributions Thanks to [@shinka](https://github.com/ShinkaM), [@oignat](https://github.com/OanaIgnat), [@andrew](https://ajyl.github.io/)
license: MIT许可证 task_categories: - 文本分类（text-classification） - 零样本分类（zero-shot-classification） - 文本生成（text-generation） language: - 英语 tags: - 医疗 - 大语言模型（LLM） - 抑郁症 - 种族 - 性别 pretty_name: HeadRoom size_categories: - 1000 < n < 10000 --- # InspAIred 数据集卡片 ## 目录 - [数据集描述](#dataset-description) - [数据集摘要](#dataset-summary) - [语言分布](#languages) - [支持任务与排行榜](#supported-tasks-and-leaderboards) - [附加信息](#additional-information) - [引用信息](#citation-information) - [贡献者](#contributions) ## 数据集描述 - **主页**: [HeadRoom 项目主页](https://github.com/MichiganNLP/depression_synthetic_data) - **代码仓库**: [HeadRoom 代码仓库](https://github.com/MichiganNLP/depression_synthetic_data) - **相关论文**: [《算法保真度：合成数据与人工生成数据中的人口统计学维度心理健康表征》](https://arxiv.org/abs/2403.16909) - **联系人**: [Shinka Mori](mailto:shinkamo@umich.edu ) ### 数据集摘要 本研究旨在探讨将GPT-3作为心理健康领域合成数据生成工具的应用价值，通过分析其**算法保真度（Algorithmic Fidelity）**——该术语由Argyle等人于2022年提出，用于指代大语言模型（LLM）拟合真实文本分布的能力。 依托GPT-3，本研究构建了HeadRoom数据集，该数据集包含3120条关于抑郁症触发应激源的合成文本，研究过程中对种族、性别以及新冠疫情前后的时间维度进行了严格控制。 本研究希望可为合成数据生成领域的研究提供助力，并帮助研究者分析与理解GPT-3模拟真实抑郁症相关文本的拟合程度。 ### 语言分布 本数据集的文本语言为英语。 ### 支持任务与排行榜 待补充 ## 附加信息 ### 引用信息 bibtex @inproceedings{mori-etal-2024-towards-algorithmic, title = "Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data", author = "Mori, Shinka and Ignat, Oana and Lee, Andrew and Mihalcea, Rada", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)", month = may, year = "2024", address = "Torino, Italy", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.1423", pages = "16378--16391", abstract = "Synthetic data generation has the potential to impact applications and domains with scarce data. However, before such data is used for sensitive tasks such as mental health, we need an understanding of how different demographics are represented in it. In our paper, we analyze the potential of producing synthetic data using GPT-3 by exploring the various stressors it attributes to different race and gender combinations, to provide insight for future researchers looking into using LLMs for data generation. Using GPT-3, we develop HeadRoom, a synthetic dataset of 3,120 posts about depression-triggering stressors, by controlling for race, gender, and time frame (before and after COVID-19). Using this dataset, we conduct semantic and lexical analyses to (1) identify the predominant stressors for each demographic group; and (2) compare our synthetic data to a human-generated dataset. We present the procedures to generate queries to develop depression data using GPT-3, and conduct analyzes to uncover the types of stressors it assigns to demographic groups, which could be used to test the limitations of LLMs for synthetic data generation for depression data. Our findings show that synthetic data mimics some of the human-generated data distribution for the predominant depression stressors across diverse demographics.", } ### 贡献者 感谢 [@shinka](https://github.com/ShinkaM)、[@oignat](https://github.com/OanaIgnat) 与 [@andrew](https://ajyl.github.io/) 为本项目做出的贡献。
数据集概述
数据集描述
数据集总结
- 名称: HeadRoom
- 描述: 使用GPT-3生成关于抑郁症触发因素的合成数据集，包含3,120条帖子，控制了种族、性别和时间框架（COVID-19前后）。
- 目的: 研究GPT-3在模拟真实生活文本分布方面的算法忠诚度，特别是在心理健康领域的应用。
语言
- 主要语言: 英语
支持的任务和排行榜
- 任务: 文本分类、零样本分类、文本生成
- 状态: 待定
附加信息
引用信息
- 论文: Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data
- 作者: Mori, Shinka; Ignat, Oana; Lee, Andrew; Mihalcea, Rada
- 出版: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- 出版商: ELRA and ICCL
- 年份: 2024
贡献者
