CarsonnnNN/TCM-Pretrain-Data-ShizhenGPT

---
license: apache-2.0
task_categories:
- text-generation
language:
- zh
size_categories:
- n>1T
tags:
- Traditional Chinese Medicine
configs:
- config_name: TCM_Book_Corpus (Text)
data_files: TCM_pretrain_book_corpus.json
- config_name: TCM_Web_Corpus (Text)
data_files: TCM_pretrain_web_corpus.jsonl
- config_name: TCM_Web_Interleaved_Data (Text & Image)
data_files: TCM_pretrain_web_vision.json
- config_name: TCM_Book_Interleaved_Data (Text & Image)
data_files: TCM_pretrain_book_vision.json
- config_name: TCM__synthesized_vision (Text & Image)
data_files: TCM_pretrain_synthesized_vision.json
---

# <span>📚 Introduction</span>

This dataset is the pre-training dataset for [ShizhenGPT](https://github.com/FreedomIntelligence/ShizhenGPT), a multimodal LLM for **Traditional Chinese Medicine (TCM)**. We open-source the largest existing TCM corpus dataset (over 5B tokens) from TCM-related websites and books. Additionally, we also open-source the largest scale TCM image-text pretraining dataset.

For details, see our [paper](https://arxiv.org/abs/2508.14706) and [GitHub repository](https://github.com/FreedomIntelligence/ShizhenGPT).

# <span>📊 Dataset Overview</span>

The open-sourced pre-training dataset consists of five parts:

| | Modality | Description | Data Quantity |
| ---------------------------------- | ------------ | ------------------------------------------------------------------------- | ------------------------------ |
| TCM\_Book\_Corpus | 📝 Text | A cleaned corpus of 3,256 TCM textbooks. | \~ 0.5 B tokens |
| TCM\_Web\_Corpus | 📝 Text | A TCM corpus collected from the web. | Over 5B tokens |
| TCM\_Book\_Interleaved\_Data | 📝 Text, 👁️ Visual | Interleaved text-image data from 306 TCM books. | 41459 entries, 50690 images |
| TCM\_Web\_Interleaved\_Data | 📝 Text, 👁️ Visual | Interleaved text-image data from the TCM web corpus. | 505465 entries, 1143954 images |
| TCM\_pretrain\_synthesized\_vision | 📝 Text, 👁️ Visual | TCM image-text pairs generated from images and their context using GPT-4o. | 144239 entries, 159534 images |

> ⚠️ Note: Due to privacy and ethical concerns, TCM signal datasets (e.g., sound and pulse) are not provided. For some signal data, refer to the [Instruction Dataset](https://huggingface.co/datasets/FreedomIntelligence/TCM-Instruction-Tuning-ShizhenGPT).

# <span>📖 Citation</span>
If you find our data useful, please consider citing our work!
```
@misc{chen2025shizhengptmultimodalllmstraditional,
title={ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine},
author={Junying Chen and Zhenyang Cai and Zhiheng Liu and Yunjin Yang and Rongsheng Wang and Qingying Xiao and Xiangyi Feng and Zhan Su and Jing Guo and Xiang Wan and Guangjun Yu and Haizhou Li and Benyou Wang},
year={2025},
eprint={2508.14706},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2508.14706},
}
```

license: apache-2.0
task_categories:
- text-generation
language:
- zh
size_categories:
- n>1T
tags:
- Traditional Chinese Medicine
configs:
- config_name: TCM_Book_Corpus (文本)
data_files: TCM_pretrain_book_corpus.json
- config_name: TCM_Web_Corpus (文本)
data_files: TCM_pretrain_web_corpus.jsonl
- config_name: TCM_Web_Interleaved_Data (文本与图像)
data_files: TCM_pretrain_web_vision.json
- config_name: TCM_Book_Interleaved_Data (文本与图像)
data_files: TCM_pretrain_book_vision.json
- config_name: TCM__synthesized_vision (文本与图像)
data_files: TCM_pretrain_synthesized_vision.json

# 📚 简介
本数据集为面向中医药（Traditional Chinese Medicine，TCM）的多模态大语言模型（Large Language Model，LLM）[ShizhenGPT](https://github.com/FreedomIntelligence/ShizhenGPT)的预训练数据集。我们开源了当前规模最大的中医药语料数据集，包含来自中医药相关网站与书籍的超50亿Token。此外，我们还开源了当前规模最大的中医药图文预训练数据集。详细信息请参阅我们的[学术论文](https://arxiv.org/abs/2508.14706)与[GitHub仓库](https://github.com/FreedomIntelligence/ShizhenGPT)。

# 📊 数据集概览
本次开源的预训练数据集包含五个部分：

| | 模态 | 描述 | 数据量 |
| ---------------------------------- | -------------- | ------------------------------------------------------------------ | ------------------------------- |
| TCM_Book_Corpus | 📝 文本 | 经过清洗的3256本中医药教科书语料库。 | 约0.5亿Token |
| TCM_Web_Corpus | 📝 文本 | 从网络收集的中医药语料库。 | 超过50亿Token |
| TCM_Book_Interleaved_Data | 📝 文本、👁️ 视觉 | 来自306本中医药书籍的交错图文数据。 | 41459条数据，50690张图像 |
| TCM_Web_Interleaved_Data | 📝 文本、👁️ 视觉 | 来自中医药网络语料库的交错图文数据。 | 505465条数据，1143954张图像 |
| TCM_pretrain_synthesized_vision | 📝 文本、👁️ 视觉 | 基于图像及其上下文通过GPT-4o生成的中医药图文配对数据。 | 144239条数据，159534张图像 |

> ⚠️ 注意：出于隐私与伦理考量，本数据集未包含中医药信号类数据集（例如脉象、声纹数据）。如需获取部分信号数据，请参阅[指令微调数据集](https://huggingface.co/datasets/FreedomIntelligence/TCM-Instruction-Tuning-ShizhenGPT)。

# 📖 引用
若您认为本数据集对您的研究有所帮助，请引用我们的工作！

@misc{chen2025shizhengptmultimodalllmstraditional,
title={ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine},
author={Junying Chen and Zhenyang Cai and Zhiheng Liu and Yunjin Yang and Rongsheng Wang and Qingying Xiao and Xiangyi Feng and Zhan Su and Jing Guo and Xiang Wan and Guangjun Yu and Haizhou Li and Benyou Wang},
year={2025},
eprint={2508.14706},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2508.14706},
}