Skin Cancer - The HAM10000 dataset

## The Original Dataset The source dataset and its full description may be accessed through the [Harvard Dataverse](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T), and should be cited as &gt;Tschandl, Philipp, 2018, "The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions", [https://doi.org/10.7910/DVN/DBW86T](https://doi.org/10.7910/DVN/DBW86T), Harvard Dataverse, V4, UNF:6:KCZFcBLiFE5ObWcTc2ZBOA== [fileUNF] ## The Current Dataset Note that the herein uploaded dataset does not contain all of the source material, namely the file `ISIC2018_Task3_Test_NatureMedicine_AI_Interaction_Benefit.tab` - which contains data on a study involving human-computer collaboration - and the folder `HAM10000_segmentations_lesion_tschandl` - containing binary segmentation masks of the training images. **Still, in contrast to most of the HAM10000 datasets published in Kaggle, the current one includes the test dataset that was curated for the [ISIC 2018 challenge (Task 3)](https://arxiv.org/abs/1902.03368).** ## Description ### Files and folders The uploaded dataset is comprised by 3 folders and 2 files, described in the table below. | Content | Type | Description | | --- | --- | --- | | `HAM10000_images_part_1` | folder | Part 1 of a set of training pictures | | `HAM10000_images_part_2` | folder | Part 2 of a set of training pictures | | `ISIC2018_Task3_Test_Images` | folder | Set of test pictures | | `HAM10000_metadata.csv` | file | Metadata associated with the training data | | `ISIC2018_Task3_Test_GroundTruth.csv` | file | Metadata associated with the test data | <br> <br> The training dataset (`HAM10000_images_part_1` and `HAM10000_images_part_2`) is called "HAM10000" meaning **"Human Against Machine with 10000 training images""** (actually 10015 images) and it corresponds to a large collection of multi-source dermatoscopic RGB images (JPG) of common pigmented skin lesions. The test dataset (`ISIC2018_Task3_Test_Images`) corresponds to 511 images. The files `HAM10000_metadata.csv` and `ISIC2018_Task3_Test_GroundTruth.csv` contain the respective metadata (data about the data) which further include other features and the labels. ### Columns of the metadata files Their structure of the metadata files follows the template presented by the table below. | Column | Type | Description| | --- | --- | --- | | `lesion_id` | String | ID of the lesion case | | `image_id` | String | ID of an image (also the name of the respective JPG file) associated with that case | | `dx` | String | Label of that case | | `dx_type` | String | Method used for diagnosing that case | | `age` | Float | Age of the person associated with that case | | `sex` | String | Sex of the person associated with that case | | `localization` | String | Location of the lesion in the person body | | `dataset` | String | Reference from which the data was taken | <br> <br> ### Values of the metadata `dx` column (the classes) The values that the column `dx` may take are tabulated below. | Value | Description | | --- | --- | | `akiec` | Actinic keratoses and intraepithelial carcinoma (also called "Bowen's disease") - an early form of **skin cancer** | | `bcc` | Basal cell carcinoma - the most common type of **skin cancer** | | `bkl` | Benign keratosis-like lesions (solar lentigines / seborrheic keratoses and lichen-planus like keratoses) - **common and benign** | | `df` | Dermatofibroma - **common and benign** | | `mel` | Melanoma - a type of **skin cancer** involving the melanin cells | | `nv` | Melanocytic nevus - the medical term for a mole (**benign**)| | `vasc` | Vascular lesions (angiomas, angiokeratomas, pyogenic granulomas and hemorrhage) (**benign**) | <br> <br> ### Values of the metadata `dx_type` column (the diagnosis methods) And the table below present the values of the column `dx_type`. | Value | Description | | --- | --- | | `histo` | Histopathology | | `follow_up` | Follow-up examination | | `consensus` | Expert consensus | | `confocal` | In-vivo confocal microscopy|数据集来源及其详尽描述可通过[哈佛数据仓库](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T)获取，并应引用如下： > Tschandl, Philipp, 2018, 'The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions', [https://doi.org/10.7910/DVN/DBW86T](https://doi.org/10.7910/DVN/DBW86T), Harvard Dataverse, V4, UNF:6:KCZFcBLiFE5ObWcTc2ZBOA== [fileUNF] 当前上传的数据集并不包含所有原始素材，具体而言，缺少文件`ISIC2018_Task3_Test_NatureMedicine_AI_Interaction_Benefit.tab`——该文件包含有关人机协作研究的资料，以及文件夹`HAM10000_segmentations_lesion_tschandl`——其中包含训练图像的二值分割掩码。然而，与Kaggle上发布的大多数HAM10000数据集相比，当前数据集包含为[ISIC 2018挑战赛（任务3）](https://arxiv.org/abs/1902.03368)精心挑选的测试数据集。 ### 文件与文件夹 上传的数据集由3个文件夹和2个文件组成，具体如下表所示。 | 内容 | 类型 | 描述 | | --- | --- | --- | | `HAM10000_images_part_1` | 文件夹 | 训练图片集的第一部分 | | `HAM10000_images_part_2` | 文件夹 | 训练图片集的第二部分 | | `ISIC2018_Task3_Test_Images` | 文件夹 | 测试图片集 | | `HAM10000_metadata.csv` | 文件 | 与训练数据关联的元数据 | | `ISIC2018_Task3_Test_GroundTruth.csv` | 文件 | 与测试数据关联的元数据 | 训练数据集（`HAM10000_images_part_1`和`HAM10000_images_part_2`）被称为`HAM10000`，意为“人类对抗机器，包含10000张训练图像”（实际上为10015张图像），对应于大量多来源的皮肤镜RGB图像（JPEG）的集合，这些图像展示了常见的色素性皮肤病变。测试数据集（`ISIC2018_Task3_Test_Images`）对应于511张图像。文件`HAM10000_metadata.csv`和`ISIC2018_Task3_Test_GroundTruth.csv`包含相应的元数据（关于数据的资料），其中进一步包括其他特征和标签。 ### 元数据文件列 元数据文件的架构遵循下表所示的模板。 | 列 | 类型 | 描述 | | --- | --- | --- | | `lesion_id` | 字符串 | 病变案例的ID | | `image_id` | 字符串 | 与该案例关联的图像ID（也是相应JPEG文件的名称） | | `dx` | 字符串 | 该案例的标签 | | `dx_type` | 字符串 | 用于诊断该案例的方法 | | `age` | 浮点数 | 与该案例关联的人的年龄 | | `sex` | 字符串 | 与该案例关联的人的性别 | | `localization` | 字符串 | 病变在人体中的位置 | | `dataset` | 字符串 | 数据来源的引用 | ### 元数据`dx`列的值（类别） `dx`列可能取的值如下表所示。 | 值 | 描述 | | --- | --- | | `akiec` | 活性角化病和上皮内癌（亦称为“鲍文病”），一种**皮肤癌**的早期形式 | | `bcc` | 基底细胞癌——最常见的**皮肤癌**类型 | | `bkl` | 良性角化病样病变（日晒斑/脂溢性角化病和扁平苔藓样角化病）——**常见且良性** | | `df` | 皮肤纤维瘤——**常见且良性** | | `mel` | 黑色素瘤——涉及黑色素细胞的**皮肤癌**类型 | | `nv` | 黑色素细胞痣——**良性**的医学术语 | | `vasc` | 血管病变（血管瘤、血管角化瘤、化脓性肉芽肿和出血）——**良性** | ### 元数据`dx_type`列的值（诊断方法） 下表呈现了`dx_type`列的值。 | 值 | 描述 | | --- | --- | | `histo` | 组织病理学 | | `follow_up` | 随访检查 | | `consensus` | 专家共识 | | `confocal` | 在体共聚焦显微镜 |