d_AbdomenAtlas1.0Mini

# Dataset Summary

One of the largest, fully-annotated CT dataset to date, including **5,195 annotated CT volumes** (with spleen, liver, kidneys, stomach,
gallbladder, pancreas, aorta, and IVC annotations).

---

# Join the AbdomenAtlas Benchmarking Project (Touchstone)

The Benchmarking Project aims to compare diverse semantic segmentation and pre-training algorithms.
We, the CCVL research group at Johns Hopkins University, invite creators of these algorithms to contribute to the initiative.
With our support, contributors will train their methodologies on the largest fully-annotated abdominal CT datasets to date.
Subsequently, we will evaluate the trained models using a large internal dataset at Johns Hopkins University.
If you are the creator of a semantic segmentation or pre-training algorithm and wish to advance medical AI by participating
in the Benchmark Project, please reach out to [e-mail]. We will provide you further details on the project
and explain your opportunities to collaborate in our future publications!

---

# Note for Touchstone Benchmarking Project

Touchstone Benchmarking Project participants should not use this dataset. The version used in the project is now in [AbdomenAtlas1.0MiniBeta](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0MiniBeta).
This version (AbdomenAtlas/AbdomenAtlas1.0Mini) was updated with improved aorta and kidney annotations.
Thus, AI algorithms trained in this dataset can be directly compared to the results in the Touchstone Project, except for these 2 organs.

---

# Dataset variants:

[AbdomenAtlas1.0Mini](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini) - 5,195 annotated CT volumes, improved label quality for aorta and kidneys

[_AbdomenAtlas1.0Mini](https://huggingface.co/datasets/AbdomenAtlas/_AbdomenAtlas1.0Mini) - same as above, but structured as large zip files, to facilitate downloading

[AbdomenAtlas1.0MiniBeta](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0MiniBeta) - 5,195 annotated CT volumes, with the noisy labels for aorta and kidneys

---

# Downloading Instructions

#### 1- Register at Huggingface, accept our terms and conditions, and create an access token:

[Create a Huggingface account](https://huggingface.co/join)

[Log in](https://huggingface.co/login)

[Accept our terms and conditions for acessing this dataset](https://huggingface.co/datasets/AbdomenAtlas/_AbdomenAtlas1.0Mini) (top of this page)

[Create a Huggingface access token](https://huggingface.co/settings/tokens) and copy it (you will use it in step 3, in paste_your_token_here)

#### 2- Install the Hugging Face library:
```bash
pip install huggingface_hub[hf_transfer]==0.24.0
HF_HUB_ENABLE_HF_TRANSFER=1
pip install ipywidgets
```
<details>
<summary style="margin-left: 25px;">[Optional] Alternative without HF Trasnsfer (slower)</summary>
<div style="margin-left: 25px;">

```bash
pip install huggingface_hub==0.24.0
```

</div>
</details>

#### 3- Download the dataset:
```bash
mkdir AbdomenAtlas
cd AbdomenAtlas
huggingface-cli download AbdomenAtlas/_AbdomenAtlas1.0Mini --token paste_your_token_here --repo-type dataset --local-dir .
```

<details>
<summary style="margin-left: 25px;">[Optional] Resume downloading</summary>
<div style="margin-left: 25px;">

In case you had a previous interrupted download, just run the huggingface-cli download command above again.
```bash
huggingface-cli download AbdomenAtlas/_AbdomenAtlas1.0Mini --token paste_your_token_here --repo-type dataset --local-dir .
```
</div>
</details>

### 4- Uncompress:
Uncompress:
```bash
bash unzip.sh
```

Check if the folder AbdomenAtlas/uncompressed contains all cases, from BDMAP_00000001 to BDMAP_00005195. If so,
you can delete the original compressed files, running:
```bash
bash delete.sh
```

---

## Paper

<b>AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks</b> <br/>
[Chongyu Qu](https://github.com/Chongyu1117)¹, [Tiezheng Zhang](https://github.com/ollie-ztz)¹, [Hualin Qiao](https://www.linkedin.com/in/hualin-qiao-a29438210/)², [Jie Liu](https://ljwztc.github.io/)³, [Yucheng Tang](https://scholar.google.com/citations?hl=en&user=0xheliUAAAAJ)⁴, [Alan L. Yuille](https://www.cs.jhu.edu/~ayuille/)¹, and [Zongwei Zhou](https://www.zongweiz.com/)^1,* <br/>
¹Johns Hopkins University, <br/>
²Rutgers University, <br/>
³City University of Hong Kong, <br/>
⁴NVIDIA <br/>
NeurIPS 2023 <br/>
[paper](https://www.cs.jhu.edu/~alanlab/Pubs23/qu2023abdomenatlas.pdf) | [code](https://github.com/MrGiovanni/AbdomenAtlas) | [dataset](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini) | [annotation](https://www.dropbox.com/scl/fi/28l5vpxrn212r2ejk32xv/AbdomenAtlas.tar.gz?rlkey=vgqmao4tgv51hv5ew24xx4xpm&dl=0) | [poster](document/neurips_poster.pdf)

<b>How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?</b> <br/>
[Wenxuan Li](https://scholar.google.com/citations?hl=en&user=tpNZM2YAAAAJ), [Alan Yuille](https://www.cs.jhu.edu/~ayuille/), and [Zongwei Zhou](https://www.zongweiz.com/)^* <br/>
Johns Hopkins University <br/>
International Conference on Learning Representations (ICLR) 2024 (oral; top 1.2%) <br/>
[paper](https://www.cs.jhu.edu/~alanlab/Pubs23/li2023suprem.pdf) | [code](https://github.com/MrGiovanni/SuPreM)

## Citation

```

@article{li2024abdomenatlas,
title={AbdomenAtlas: A large-scale, detailed-annotated, \& multi-center dataset for efficient transfer learning and open algorithmic benchmarking},
author={Li, Wenxuan and Qu, Chongyu and Chen, Xiaoxi and Bassi, Pedro RAS and Shi, Yijia and Lai, Yuxiang and Yu, Qian and Xue, Huimin and Chen, Yixiong and Lin, Xiaorui and others},
journal={Medical Image Analysis},
pages={103285},
year={2024},
publisher={Elsevier},
url={https://github.com/MrGiovanni/AbdomenAtlas}
}

@article{bassi2024touchstone,
title={Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?},
author={Bassi, Pedro RAS and Li, Wenxuan and Tang, Yucheng and Isensee, Fabian and Wang, Zifu and Chen, Jieneng and Chou, Yu-Cheng and Kirchhoff, Yannick and Rokuss, Maximilian and Huang, Ziyan and others},
journal={arXiv preprint arXiv:2411.03670},
year={2024}
}

@article{qu2023abdomenatlas,
title={Abdomenatlas-8k: Annotating 8,000 CT volumes for multi-organ segmentation in three weeks},
author={Qu, Chongyu and Zhang, Tiezheng and Qiao, Hualin and Tang, Yucheng and Yuille, Alan L and Zhou, Zongwei},
journal={Advances in Neural Information Processing Systems},
volume={36},
year={2023}
}

@inproceedings{li2024well,
title={How Well Do Supervised Models Transfer to 3D Image Segmentation?},
author={Li, Wenxuan and Yuille, Alan and Zhou, Zongwei},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024}
}
```

## Acknowledgements
This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research and partially by the Patrick J. McGovern Foundation Award. We appreciate the effort of the MONAI Team to provide open-source code for the community.

## License
AbdomenAtlas 1.0</a> is licensed under <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1" target="_blank" rel="license noopener noreferrer" style="display:inline-block;">CC BY-NC-SA 4.0.</a></p>

## Uploading AbdomenAtlas to HuggingFace
The file AbdomenAtlasUploadMultipleFolders.ipynb has the code we used to upload AbdomenAtlas to Hugging Face. It may be ncessary to run the script multiple times, until it finishes without an uploading error. The uploading script requires PyTorch, huggingface_hub, and Jupyter Notebook.

# 数据集摘要

目前已公开的规模最大的全标注计算机断层扫描（CT）数据集之一，包含**5195份经标注的CT容积数据**，标注器官涵盖脾脏、肝脏、肾脏、胃、胆囊、胰腺、主动脉以及下腔静脉（inferior vena cava, IVC）。

---

# 加入AbdomenAtlas基准项目（Touchstone）

本基准项目旨在对各类语义分割与预训练算法开展对比评测。我们，约翰·霍普金斯大学CCVL研究团队，诚邀各类算法的开发者参与本项目。依托本团队提供的支持，参与者可基于目前已公开的规模最大的全标注腹部CT数据集训练其算法框架。后续，我们将依托约翰·霍普金斯大学内部的大型数据集对训练完成的模型进行评测。若您是语义分割或预训练算法的开发者，希望通过参与本基准项目推动医学人工智能的发展，请致信[e-mail]。我们将为您提供项目的详细信息，并告知您在本团队后续发表论文中参与合作的机会！

---

# Touchstone基准项目注意事项

**Touchstone基准项目参与者请勿使用本数据集**。本项目所使用的数据集版本现已迁移至[AbdomenAtlas1.0MiniBeta](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0MiniBeta)。当前的[AbdomenAtlas1.0Mini](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini)版本已针对主动脉与肾脏的标注进行了优化更新。因此，基于本数据集训练的人工智能算法，除这两类器官外，其评测结果可直接与Touchstone项目中的结果进行对比。

---

# 数据集变体

- [AbdomenAtlas1.0Mini](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini)：包含5195份经标注的CT容积数据，优化了主动脉与肾脏的标注质量
- [_AbdomenAtlas1.0Mini](https://huggingface.co/datasets/AbdomenAtlas/_AbdomenAtlas1.0Mini)：与上述版本一致，但以大型ZIP压缩包的形式组织，便于下载
- [AbdomenAtlas1.0MiniBeta](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0MiniBeta)：包含5195份经标注的CT容积数据，主动脉与肾脏的标注存在噪声

---

# 下载指南

#### 1. 注册Hugging Face账号，同意数据集使用条款并创建访问令牌：

[注册Hugging Face账号](https://huggingface.co/join)
[登录账号](https://huggingface.co/login)
[同意本数据集的访问条款](https://huggingface.co/datasets/AbdomenAtlas/_AbdomenAtlas1.0Mini)（本页面顶部）
[创建Hugging Face访问令牌](https://huggingface.co/settings/tokens)并复制（您将在步骤3中使用该令牌，替换`paste_your_token_here`）

#### 2. 安装Hugging Face相关库：
bash
pip install huggingface_hub[hf_transfer]==0.24.0
HF_HUB_ENABLE_HF_TRANSFER=1
pip install ipywidgets

<details>
<summary style="margin-left: 25px;">[可选] 不使用HF Transfer的替代方案（速度较慢）</summary>
<div style="margin-left: 25px;">

bash
pip install huggingface_hub==0.24.0

</div>
</details>

#### 3. 下载数据集：
bash
mkdir AbdomenAtlas
cd AbdomenAtlas
huggingface-cli download AbdomenAtlas/_AbdomenAtlas1.0Mini --token paste_your_token_here --repo-type dataset --local-dir .

<details>
<summary style="margin-left: 25px;">[可选] 续传下载</summary>
<div style="margin-left: 25px;">

若您此前的下载中断，只需重新运行上述的`huggingface-cli download`命令即可。
bash
huggingface-cli download AbdomenAtlas/_AbdomenAtlas1.0Mini --token paste_your_token_here --repo-type dataset --local-dir .

</div>
</details>

#### 4. 解压文件：

执行解压命令：
bash
bash unzip.sh

请检查`AbdomenAtlas/uncompressed`文件夹是否包含全部样本（从BDMAP_00000001至BDMAP_00005195）。若确认无误，可运行以下命令删除原始压缩文件：
bash
bash delete.sh

---

## 相关论文

**《AbdomenAtlas-8K：三周内完成8000份CT容积数据的多器官分割标注》** <br/>
[瞿崇宇](https://github.com/Chongyu1117)¹、[张铁铮](https://github.com/ollie-ztz)¹、[乔华林](https://www.linkedin.com/in/hualin-qiao-a29438210/)²、[刘杰](https://ljwztc.github.io/)³、[唐玉成](https://scholar.google.com/citations?hl=en&user=0xheliUAAAAJ)⁴、[Alan L. Yuille](https://www.cs.jhu.edu/~ayuille/)¹、[周宗伟](https://www.zongweiz.com/)^1,* <br/>
¹约翰·霍普金斯大学 <br/>
²罗格斯大学 <br/>
³香港城市大学 <br/>
⁴英伟达（NVIDIA） <br/>
NeurIPS 2023 <br/>
[论文](https://www.cs.jhu.edu/~alanlab/Pubs23/qu2023abdomenatlas.pdf) | [代码](https://github.com/MrGiovanni/AbdomenAtlas) | [数据集](https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini) | [标注数据](https://www.dropbox.com/scl/fi/28l5vpxrn212r2ejk32xv/AbdomenAtlas.tar.gz?rlkey=vgqmao4tgv51hv5ew24xx4xpm&dl=0) | [海报](document/neurips_poster.pdf)

**《监督式三维模型在医学影像任务中的迁移性能如何？》** <br/>
[李文轩](https://scholar.google.com/citations?hl=en&user=tpNZM2YAAAAJ)、[Alan Yuille](https://www.cs.jhu.edu/~ayuille/)、[周宗伟](https://www.zongweiz.com/)^* <br/>
约翰·霍普金斯大学 <br/>
国际学习表征会议（ICLR 2024）（口头报告；Top 1.2%） <br/>
[论文](https://www.cs.jhu.edu/~alanlab/Pubs23/li2023suprem.pdf) | [代码](https://github.com/MrGiovanni/SuPreM)

## 引用格式

@article{li2024abdomenatlas,
title={AbdomenAtlas: A large-scale, detailed-annotated, & multi-center dataset for efficient transfer learning and open algorithmic benchmarking},
author={Li, Wenxuan and Qu, Chongyu and Chen, Xiaoxi and Bassi, Pedro RAS and Shi, Yijia and Lai, Yuxiang and Yu, Qian and Xue, Huimin and Chen, Yixiong and Lin, Xiaorui and others},
journal={Medical Image Analysis},
pages={103285},
year={2024},
publisher={Elsevier},
url={https://github.com/MrGiovanni/AbdomenAtlas}
}

@article{bassi2024touchstone,
title={Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?},
author={Bassi, Pedro RAS and Li, Wenxuan and Tang, Yucheng and Isensee, Fabian and Wang, Zifu and Chen, Jieneng and Chou, Yu-Cheng and Kirchhoff, Yannick and Rokuss, Maximilian and Huang, Ziyan and others},
journal={arXiv preprint arXiv:2411.03670},
year={2024}
}

@article{qu2023abdomenatlas,
title={Abdomenatlas-8k: Annotating 8,000 CT volumes for multi-organ segmentation in three weeks},
author={Qu, Chongyu and Zhang, Tiezheng and Qiao, Hualin and Tang, Yucheng and Yuille, Alan L and Zhou, Zongwei},
journal={Advances in Neural Information Processing Systems},
volume={36},
year={2023}
}

@inproceedings{li2024well,
title={How Well Do Supervised Models Transfer to 3D Image Segmentation?},
author={Li, Wenxuan and Yuille, Alan and Zhou, Zongwei},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024}
}

## 致谢

本研究得到了Lustgarten胰腺癌研究基金会以及Patrick J. McGovern基金会奖项的部分资助。我们感谢MONAI团队为社区提供开源代码的辛勤工作。

## 许可协议

AbdomenAtlas 1.0采用<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1" target="_blank" rel="license noopener noreferrer" style="display:inline-block;">CC BY-NC-SA 4.0</a>许可协议。

## 上传至Hugging Face

我们用于将AbdomenAtlas上传至Hugging Face的代码位于`AbdomenAtlasUploadMultipleFolders.ipynb`文件中。由于上传可能存在失败情况，需多次运行该脚本直至上传成功。本上传脚本依赖PyTorch、huggingface_hub以及Jupyter Notebook环境。