Derm1M

# Dataset Card for Derm1M

<div align="center">
<img src="https://raw.githubusercontent.com/SiyuanYan1/Derm1M/main/assets/ICCV_Derm1M_poster.png"
alt="Derm1M Overview"
width="800" />
</div>

<p align="center">
<strong>Paper:</strong> <a href="https://arxiv.org/abs/2503.14911" target="_blank">ArXiv</a>
&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;&nbsp;
<strong>Code:</strong> <a href="https://github.com/SiyuanYan1/Derm1M" target="_blank">GitHub</a>
&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;&nbsp;
<strong>Models:</strong> <a href="https://huggingface.co/redlessone/DermLIP_ViT-B-16" target="_blank">DermLIP-ViT-B-16</a> | <a href="https://huggingface.co/redlessone/DermLIP_PanDerm-base-w-PubMed-256" target="_blank">DermLIP-PanDerm</a>
</p>

## Dataset Summary

**Derm1M** is a large-scale, million-scale vision-language dataset for dermatology containing **1,029,761 dermatological image-text pairs** from **403,563 unique images**. The dataset covers **390 skin conditions** organized in a four-level expert ontology and includes **130 clinical concepts**. With rich contextual captions averaging 41 tokens, Derm1M enables explainable multimodal learning, zero-shot and few-shot diagnosis, cross-modal retrieval, and visual question answering in clinical dermatology settings.

This dataset is **257× larger** than any previous dermatology vision-language corpus and is specifically designed for training and evaluating vision-language models in the dermatology domain.

## Dataset Details
Derm1M provides comprehensive annotations including:
- **1,029,761 image-text pairs** with detailed clinical captions
- **390 skin conditions** structured in a hierarchical ontology
- **130 clinical concepts** extracted per image
- **Rich metadata** including image sources, clinical contexts, and ontological relationships
- **Structured ontology** in JSON format for hierarchical disease understanding

### Dataset Description
- **Curated by:** Siyuan Yan, Ming Hu, Yiwen Jiang, Xieji Li
- **Language(s):** English
- **License:** CC BY-NC 4.0 (Non-Commercial Use Only)
- **Supported Tasks:**
- Vision-language pre-training
- Zero-shot classification
- Few-shot learning
- Cross-modal retrieval
- Concept annotation/explanation
- Visual question answering

### Dataset Sources

- **Repository:** https://github.com/SiyuanYan1/Derm1M
- **Paper:** https://arxiv.org/abs/2503.14911
- **Models:**
- [DermLIP-ViT-B-16](https://huggingface.co/redlessone/DermLIP_ViT-B-16)
- [DermLIP-PanDerm-base-w-PubMed-256](https://huggingface.co/redlessone/DermLIP_PanDerm-base-w-PubMed-256)

## Dataset Structure
```
dataset_root/
├── xxx/ # unzip all zip files
├── Derm1M_v2_pretrain.csv # text + meta per image for model pretraining
├── Derm1M_v2_validation.csv # text + meta per image for model validation
├── concept.csv # extracted concept annotations per image
├── ontology.json # skin disease hierarchy
```
### Data Instances
```python
{
'filename': 'image_001.jpg',
'truncated_caption': 'Clinical photograph showing erythematous papules and pustules on facial skin, consistent with inflammatory acne...',
'disease_label': 'Acne Vulgaris',
'hierarchical_disease_label': 'Inflammatory Skin Diseases, Acne and Related Disorders, Acne Vulgaris'
'skin_concept': 'erythema, papule, pustule, facial_distribution',
'source': 'pubmed',
'source_type': 'knowledge',
.......
}
```
## Citation

```
@misc{yan2025derm1m,
title = {Derm1M: A Million‑Scale Vision‑Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology},
author = {Siyuan Yan and Ming Hu and Yiwen Jiang and Xieji Li and Hao Fei and Philipp Tschandl and Harald Kittler and Zongyuan Ge},
year = {2025},
eprint = {2503.14911},
archivePrefix= {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2503.14911}
}

@article{yan2025multimodal,
title={A multimodal vision foundation model for clinical dermatology},
author={Yan, Siyuan and Yu, Zhen and Primiero, Clare and Vico-Alonso, Cristina and Wang, Zhonghua and Yang, Litao and Tschandl, Philipp and Hu, Ming and Ju, Lie and Tan, Gin and others},
journal={Nature Medicine},
pages={1--12},
year={2025},
publisher={Nature Publishing Group}
}
```

# Derm1M 数据集卡片

<div align="center">
<img src="https://raw.githubusercontent.com/SiyuanYan1/Derm1M/main/assets/ICCV_Derm1M_poster.png"
alt="Derm1M 概览"
width="800" />
</div>

<p align="center">
<strong>论文：</strong> <a href="https://arxiv.org/abs/2503.14911" target="_blank">ArXiv</a>
&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;&nbsp;
<strong>代码：</strong> <a href="https://github.com/SiyuanYan1/Derm1M" target="_blank">GitHub</a>
&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;&nbsp;
<strong>模型：</strong> <a href="https://huggingface.co/redlessone/DermLIP_ViT-B-16" target="_blank">DermLIP-ViT-B-16</a> | <a href="https://huggingface.co/redlessone/DermLIP_PanDerm-base-w-PubMed-256" target="_blank">DermLIP-PanDerm</a>
</p>

## 数据集概述
**Derm1M** 是一款面向皮肤病领域的大规模百万级视觉语言数据集，包含**1,029,761 对皮肤病图像-文本对**，源自**403,563 张唯一图像**。该数据集覆盖**390 种皮肤病症**，采用四级专家本体进行层级组织，并包含**130 个临床概念**。其配套的上下文标注平均包含 41 个 Token，可支持可解释性多模态学习、零样本（Zero-shot）与少样本（Few-shot）皮肤病诊断、跨模态检索，以及临床皮肤病场景下的视觉问答任务。

该数据集规模较此前所有皮肤病视觉语言语料库**大 257 倍**，专为皮肤病领域视觉语言模型的训练与评估设计。

## 数据集详情
Derm1M 提供了全面的标注信息，包括：
- **1,029,761 对图像-文本对**，附带详细的临床标注文本
- **390 种皮肤病症**，采用层级本体结构组织
- **130 个临床概念**，为每张图像单独提取
- **丰富的元数据**，包括图像来源、临床背景与本体关联关系
- **JSON 格式的结构化本体**，用于层级化疾病理解

### 数据集描述
- **数据整理者：** Siyuan Yan、Ming Hu、Yiwen Jiang、Xieji Li
- **语言：** 英语
- **许可协议：** CC BY-NC 4.0（仅允许非商业用途）
- **支持任务：**
- 视觉语言预训练
- 零样本分类
- 少样本学习
- 跨模态检索
- 概念标注/解释
- 视觉问答

### 数据集来源
- **代码仓库：** https://github.com/SiyuanYan1/Derm1M
- **学术论文：** https://arxiv.org/abs/2503.14911
- **模型：**
- [DermLIP-ViT-B-16](https://huggingface.co/redlessone/DermLIP_ViT-B-16)
- [DermLIP-PanDerm-base-w-PubMed-256](https://huggingface.co/redlessone/DermLIP_PanDerm-base-w-PubMed-256)

## 数据集结构

dataset_root/
├── xxx/ # 解压所有压缩文件
├── Derm1M_v2_pretrain.csv # 用于模型预训练的单图像文本与元数据
├── Derm1M_v2_validation.csv # 用于模型验证的单图像文本与元数据
├── concept.csv # 单图像提取的概念标注
├── ontology.json # 皮肤疾病层级本体

### 数据实例
python
{
'filename': 'image_001.jpg',
'truncated_caption': 'Clinical photograph showing erythematous papules and pustules on facial skin, consistent with inflammatory acne...',
'disease_label': 'Acne Vulgaris',
'hierarchical_disease_label': 'Inflammatory Skin Diseases, Acne and Related Disorders, Acne Vulgaris',
'skin_concept': 'erythema, papule, pustule, facial_distribution',
'source': 'pubmed',
'source_type': 'knowledge',
.......
}

## 引用

@misc{yan2025derm1m,
title = {Derm1M: A Million‑Scale Vision‑Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology},
author = {Siyuan Yan and Ming Hu and Yiwen Jiang and Xieji Li and Hao Fei and Philipp Tschandl and Harald Kittler and Zongyuan Ge},
year = {2025},
eprint = {2503.14911},
archivePrefix= {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2503.14911}
}

@article{yan2025multimodal,
title={A multimodal vision foundation model for clinical dermatology},
author={Yan, Siyuan and Yu, Zhen and Primiero, Clare and Vico-Alonso, Cristina and Wang, Zhonghua and Yang, Litao and Tschandl, Philipp and Hu, Ming and Ju, Lie and Tan, Gin and others},
journal={Nature Medicine},
pages={1--12},
year={2025},
publisher={Nature Publishing Group}
}

搜集汇总

背景与挑战

背景概述

Derm1M是一个百万规模的皮肤病学视觉-语言数据集，包含1,029,761个图像-文本对，覆盖390种皮肤病症和130个临床概念，支持多种医疗AI任务。该数据集是目前该领域最大的数据集之一，专为皮肤病学领域的视觉-语言模型训练和评估设计。