A Multi-Modal Deep Learning Framework for Classifying Melanoma and Non-Melanoma Skin Cancers Using Dermoscopic Images and Clinical Metadata

Efficient and timely identification of skin cancer, particularly non-melanoma and melanoma varieties such as Squamous Cell Carcinoma (SCC) and Basal Cell Carcinoma (BCC), is essential for dependable, efficient treatment and improved patient outcomes. On the other hand, traditional diagnostic techniques rely mostly on dermoscopic interpretation during image collection, which can be subjective and prone to inaccuracy. This research proposes a Multi-Modal Deep Learning (MMDL) framework that works with clinical metadata and dermoscopic pictures that include information such as age, sex, and lesion location to improve skin cancer classification in order to assess such promising challenges. Convolutional Neural Networks (CNNs) are used in this suggested soliton's dual branch design to process visual features. Organized, aligned, and structured metadata is thereby encoded using a parallel Multilayer Perceptron (MLP). In order to get complementary information, feature fusion is done later. Nevertheless, the HAM10000 dataset is used to train and analyze the suggested solution, which yields an overall accuracy of 93.4%, an F1 score of 91.7%, specificity of 93.6%, and sensitivity of 90.5% across several classes. Melanoma, benign lesions, SCC, and BCC are among them. Analysis of the proposed solution showed increased classification performance by an average of 5-8% across key parameters when compared to single-modality methods. In addition to its clinical implications, this study touches on the ethical importance of enhancing skin cancer detection accuracy, especially when decisions impact patient dignity, informed care, and the responsible use of melanoma and non-melanoma skin cancer.