SNC_Net: Skin Cancer Detection by Integrating Handcrafted and Deep Learning-Based Features Using Dermoscopy Images

Abstract

The medical sciences are facing a major problem with the auto-detection of disease due to the fast growth in population density. Intelligent systems assist medical professionals in early disease detection and also help to provide consistent treatment that reduces the mortality rate. Skin cancer is considered to be the deadliest and most severe kind of cancer. Medical professionals utilize dermoscopy images to make a manual diagnosis of skin cancer. This method is labor-intensive and time-consuming and demands a considerable level of expertise. Automated detection methods are necessary for the early detection of skin cancer. The occurrence of hair and air bubbles in dermoscopic images affects the diagnosis of skin cancer. This research aims to classify eight different types of skin cancer, namely actinic keratosis (AKs), dermatofibroma (DFa), melanoma (MELa), basal cell carcinoma (BCCa), squamous cell carcinoma (SCCa), melanocytic nevus (MNi), vascular lesion (VASn), and benign keratosis (BKs). In this study, we propose SNC_Net, which integrates features derived from dermoscopic images through deep learning (DL) models and handcrafted (HC) feature extraction methods with the aim of improving the performance of the classifier. A convolutional neural network (CNN) is employed for classification. Dermoscopy images from the publicly accessible ISIC 2019 dataset for skin cancer detection is utilized to train and validate the model. The performance of the proposed model is compared with four baseline models, namely EfficientNetB0 (B1), MobileNetV2 (B2), DenseNet-121 (B3), and ResNet-101 (B4), and six state-of-the-art (SOTA) classifiers. With an accuracy of 97.81%, a precision of 98.31%, a recall of 97.89%, and an F1 score of 98.10%, the proposed model outperformed the SOTA classifiers as well as the four baseline models. Moreover, an Ablation study is also performed on the proposed method to validate its performance. The proposed method therefore assists dermatologists and other medical professionals in early skin cancer detection.

1. Introduction

Skin cancer is the most prevalent form of cancer. Clinical screening initiates the diagnostic procedure that is followed by a histological examination such as dermoscopy and biopsy [1]. Skin cancer is ultimately caused by DNA mutations that disrupt the normal growth of skin cells, resulting in cancer. Ultra Violet (UV) radiations are the significant contributor to the development of skin cancer. In addition, fair complexion, exposure to chemicals and radiation, old age, smoking and severe skin trauma or burns contribute to the formation of skin cancer [2]. Skin cancer is the most frequent kind of cancer diagnosed in the United States. Among the different classes of skin cancer, melanoma is considered the deadliest form of skin cancer [3]. Melanoma is identified in less than 1% of all cases of skin cancer, but it is considered the primary cause of mortality associated with skin cancer [4]. According to the American Cancer Society’s predictions, there will be approx. 110,213 new cases of melanoma and 7560 deaths from the disease in the US alone in 2024 [5]. When melanoma spreads to the lymph nodes, its five-year survival rate is almost 66%, whereas the survival rate drops to 27% when the cancer spreads to other organs [6]. The survival rate of a patient is almost 99 percent when melanoma is detected in the initial stage [7]. The term non-melanoma cancers encompass a diverse range of subtypes of skin cancer. Non-melanoma skin malignancies include basal cell carcinoma (BCCa), squamous cell carcinoma (SCCa), and Merkel cell carcinoma (MCC). It is generally accepted that the prognoses for skin cancers other than melanoma are less harmful [8]. SCCa and BCCa are the most occurring types after melanoma, but these diseases are less aggressive than melanoma [9]. Dermoscopy is a non-surgical technique employed by doctors to identify suspicious skin lesions [10]. Dermatologists examine the affected area of the skin for any irregularities in color, size, shape, texture, or border that may indicate the presence of skin cancer [11]. It is utilized to magnify the lesion’s location, thus allowing the dermatologist to more closely examine the structure. Moreover, it is difficult to detect the various types of skin cancer accurately; therefore, it is necessary for the dermatologist to have experience in this field [12]. The manual examination accuracy rates range from 50% to 60% for highly skilled dermatologists, which is concerning [13]. An incorrect positive diagnosis needs a biopsy, whereas a mistaken negative diagnosis does not need a biopsy and the skin cancer remains undiagnosed, and the individual may die as a result of the wrong diagnosis [14]. The emergence of noise in dermoscopic photos, such as hair, air bubbles, and other artifacts, together with lighting effects, complicates the identification of skin cancer [15]. This demands the development of an AI system that can reliably and autonomously identify various types of skin cancer from dermoscopic images [16]. Pre-processing is the first and most important stage in creating an automated detection system. Moreover, the dermoscopic image quality is enhanced by the elimination of numerous artifacts, such as hair and other artifacts that create hindrances in the diagnosis process [17]. In order to remove hairs, scientists developed a variety of pre-processing techniques based on morphological operations and contrast enhancement. Image inpainting is also used to modify the values of the hair pixels to those of neighboring pixels [18].

The ABCD rule is one of the most common methods for identifying features in dermoscopic images. Four parameters are taken into account in this approach: the skin lesion diameter, color change, border structure, and asymmetry [19,20]. The seven-point checklist is another common tool for the detection of skin cancer [21].

The automated identification of skin cancer by medical imaging improves a dermatologist’s clinical practice. Due to the inherent complexities of the field of dermatology, scientists are compelled to concentrate their work on the advancement and implementation of artificial intelligence (AI)-based technologies intended for the detection of skin cancer. The utilization of AI for the purpose of classifying skin cancer has received significant attention recently. Researchers have achieved significant progress, predominantly in the detection of disease patterns in medical imaging, through the implementation of AI [17]. In the field of dermatology, AI tools and apps are currently being developed with the goal of determining the severity of various diseases [18]. These AI tools are an evolution of a computer algorithm that can learn on its own and carry out certain tasks related to dermatology, such as distinguishing between different types of skin cancer lesions [19,20].

Moreover, HC features are used to extract features from numerous shapes, textures, sizes, and colors, which are used to detect skin cancer [22]. Textural attributes provide information about the spatial distribution of tone variations in a given area as well as the connection between neighboring pixels. Texture is regarded as a crucial element since it is considered both perceptive and practical [23]. The effectiveness of the image content and visual appearance depends on how well it uses color. The color features of an image are extracted using color histograms [24]. Currently, researchers successfully retrieve and classify features from dermoscopic photos using deep learning (DL) algorithms. CNN uses its deep layers as feature extractors [25]. CNN features are trained using training data, whereas experts craft HC features to determine a particular set of attributes [26]. The need to compute resources and the accessibility of huge training sets are the primary barriers to using a CNN as an effective feature extractor [27].

Apart from the traditional ways of classifying dermoscopic images solely based on features obtained by HC or DL techniques, researchers have now started classifying dermoscopic images utilizing hybrid features [28]. In order to generate improved feature vectors that improve the performance of the classifier, hybrid approaches blend high-level qualities obtained via expert consultation with low-level features retrieved using DL methods. This leads to a more detailed explanation of the image [29].

To improve the performance of the classifier, we used inception V3 (DL method) and HC features to extract features from dermoscopic images. Using the advantages of both DL and HC feature extraction techniques, this method aims to extract prominent features. The proposed model consists of three steps. In the first step, the pre-processing of dermoscopy images is completed. In the next step, feature extraction and fusion are carried out, and classification is performed in the last stages. In addition, HC and inception V3 are used for feature extraction; then, feature fusion is used to combine these features, and lastly, CNN is used to perform multiclassification using ISIC 2019. The main conclusions drawn from this study are as follows:

• A variety of morphological operations are executed in the pre-processing phase to remove hair and anomalies from dermoscopy images to enhance image quality.

• To improve the accuracy of dermoscopy-based skin cancer diagnosis, we apply HC methods and Inception V3 for efficient feature extraction and use convolutional neural networks (CNNs) for classification.

• The problems related to class imbalance in ISIC 2019 are effectively addressed by employing the SMOTE Tomek.

• A comprehensive evaluation of the performance of the proposed model is carried out through a comparison of evaluation metrics with the results of four baseline classifiers, namely EfficientNetB0 (B1) [30], MobileNetV2 (B2) [31], DenseNet-121 (B3) [32], and ResNet-101 (B4) [33], and SOTA classifiers. The results indicate that the effectiveness of the proposed model is superior when compared to other modern models.

• The most significant visual features of various skin cancer classes are identified using the Grad-CAM heat map method.

• An innovative framework is developed to diagnose and classify various types of skin cancers in patients by utilizing dermoscopy images.

• An ablation study is performed to evaluate the practicality of the proposed model.

This paper is organized as follows: Section 2 provides a brief overview of the most recent methods for the detection of skin cancer. Section 4 offers a brief description of the data pre-processing, HC techniques, and InceptionV3 used in this study. The experimentation and the results and discussions are presented in Section 5. The limitations of this study are discussed in Section 6. Section 7 covers the conclusion as well as future work.

2. Related Work

In recent years, numerous methodologies and techniques have been proposed by researchers to automate the detection of skin cancer using dermoscopy images. Through the utilization of feature extraction methodologies, these approaches can be classified as follows: (a) a fusion of HC and deep learning methods; (b) handcrafted methods; and (c) deep learning [34]. Table 1 provides a brief summary of the literature. Moreover, comprehensive analyses of these methodologies are presented in the subsequent sections.

2.1. Handcrafted Features

Researchers utilize different combinations of HC features, including texture, color, border, and shape, in order to differentiate dermoscopy images that depict malignant lesions from benign lesions. Bakheet et al. [35] suggested that an SVM framework could be utilized to detect the presence of malignant melanoma. This framework includes a set of HOG features that have been optimized. During the process of HOG feature extraction, orientation histograms are generated to represent the gradient of the local image at every pixel within a lesion region with the purpose of computing local features. Khan et al. [36] introduced a computer-assisted approach for noise elimination in dermoscopy images. A Gaussian filter was incorporated by the researchers during the pre-processing phase. In order to perform segmentation, the K-mean algorithm must be implemented. To aid in the identification of melanoma, color attributes, in addition to local and global textural characteristics, are extracted in RGB space from the skin lesion. Warsi et al. [37] presented an innovative methodology that aims to streamline the process of feature extraction from dermoscopy images. The approach utilizes a backpropagation multilayer neural network (NN) for the purpose of classifying extracted features as color-texture features (CTFs) in three dimensions. In their publication, Kumar et al. [38] introduced an innovative methodology for the detection of skin cancer. For segmentation, the system employs a fuzzy C-means clustering algorithm. To train the classifier, color and local and global textural features are extracted. To improve melanoma detection, Akan et al. [39] suggested the implementation of a novel feature descriptor (NC) based on the sample’s pigmentation abundance. The utilization of threshold values for the number of color differences (NCD), which is an innovative attribute in the classification of skin lesions, is required for its computation.

2.2. Deep Learning and Handcrafted Feature Fusion

Codella et al. [40] introduced a system for the detection of melanoma. Several HC features, including a multi-scale variant color histogram, an edge histogram, and a color histogram, are incorporated into the system. Additionally, the system employs sparse coding strategies and features derived from multiple DLMs, namely a deep residual network, a Caffe convolutional neural network, and a fully convolutional U-Net. With the application of the extracted properties, both the segmented skin lesion and the entire image were incorporated into a specially delineated region. Li et al. [41] created a two-step process to classify dermoscopy images of the ISIC 2018 dataset by integrating HC and DL features based on clinical criteria. The segmentation of lesions is achieved through the implementation of an enhanced UNet. For the extraction of global features, DenseNet201 and ResNet50V2 DLMs are utilized. Morphological, pigment, and textural attributes are a few examples of HC features. The ideal ensemble parameters are determined by combining these features using LightGBM. Khan et al. [42] proposed a hybrid approach for cutaneous lesion identification, which integrates a deep CNN with an optimized color feature (OCF). Lesions are divided using the procedure that was previously described. The technique examination incorporates three datasets. Almaraz et al. [43] recommend the utilization of a computer-aided design (CAD) system for the classification of dermoscopy images using the ISIC 2018 dataset. The aforementioned approach combines DL characteristics that utilize Mutual Information (MI) measurements with HC characteristics that are associated with the ABCD rule. A variety of classifiers are employed to accomplish classification tasks, such as support vector machines (SVMs), Relevant Vector Machines (RVMs), and linear regression (LR). Jayapriya et al. [44] devised a two-stage framework with the objective of detecting and categorizing skin lesions for the purpose of melanoma diagnosis. For segmentation, the VGG16 and GoogLeNet models are implemented, whereas for classification, the deep residual network and HC techniques are utilized to integrate features. For data analysis, the ISBI 2016 and ISIC 2017 datasets are utilized. In order to classify the data, Kumar et al. [45] mixed a CNN and neural networks in a hybrid way. A neural network is supplied with features extracted from HOSMI-LBP to aid in the process of categorization. Additionally, CNN facilitates the direct classification of images. In determining the ultimate result, the sum of the NN and CNN’s contributions is calculated. The pre-processing step conceals crucial information associated with the border, shape, and texture of the skin lesion, which, in turn, affects the visibility of the information in dermoscopy images. The implementation of this approach enhances the overall precision of feature extraction and classification.

2.3. Deep Learning Features

DL methods are increasingly being utilized by experts in medical imaging as a result of their dual functionality as classifiers and feature extractors. Salido et al. [46] eliminated hair from the images of the PH2 dataset through pre-processing prior to classifying and extracting features with AlexNet. Four categories of skin cancer were categorized by Dorj et al. [47] by applying ECOC SVM with AlexNet. The photos included in the ISIC 2018 dataset were classified by Shahin et al. [48] utilizing a fusion of Inception V3 and ResNet-50. Dascalu et al. [49] used sound analysis and deep learning techniques for the diagnosis of skin cancer. Dermoscopy images from the ISIC 2017 dataset were segmented by Pham et al. [50] utilizing several segmentation techniques, such as U-Net, U-Net ensemble, and U-Net with a VGG16 encoder. As classifiers, the deep CNNs DenseNet-161, ResNet-152, and Inception-v4 are applied to segmented images. It was discovered that DenseNet-161 achieves the highest accuracy rate of 86%. Bisla et al. [51] created a DL technique for data augmentation and cleansing. The statistical examination conducted by Brinker et al. [52] revealed that the DL network outperformed the dermatologists’ classification. The HAM10000 dataset was classified by Carcagn et al. [53] by utilizing SVM and a DenseNet architecture. In order to improve categorization, innovative multilayer fine-tuning methods and specialized network architecture provide exceptionally discriminative features. Sarkar et al. [54] employed a deep depthwise convolutional technique for the binary classification of disease. Melanoma was automatically identified by El-Khatib et al. [55] utilizing DL-based techniques. In their study, Adegun et al. [56] introduced a DenseNet framework that utilized FCN to classify and segment skin lesions. In order to assist in the localization of lesion boundaries and the refinement of contours, the system integrates a CRF module. This module employs paired edge potentials, which are generated by linearly combining Gaussian kernels. A comprehensive examination of the application of DLMs in binary classification for the detection of malignant melanoma was conducted by Naeem et al. [27]. In their 2021 study, Kumar et al. [57] employed DL methods to determine the incidence of skin cancer. Ali et al. [58] developed a deep CNN and denoised the HAM10000 dataset to eliminate unwanted features, such as air bubbles and artifacts, with the intention of diagnosing skin cancer. Sevli et al. [59] presented a CNN model that can detect the seven distinct forms of skin lesions found in the HAM10000 dataset. Acosta et al. [60] presented the two-step categorization approach in which a reduced skin lesion is classified utilizing ResNet152 after the Mask R_CNN-based skin lesion classifier. In the study by Brinker et al. [61], the efficiency of dermatologists at various stages of the hierarchy was compared to the DL algorithm that was trained to employ open-source images. A CNN is more precise than dermatologists in detecting melanoma.

Khan et al. [62] proposed a method that employs the decorrelation approach to pre-process dermoscopy images prior to delivering them to the MASK-RCNN for lesion segmentation. At present, the segmented RGB images extracted from the ground truth images of the ISBI2016 and ISIC2017 datasets are being utilized to train the MASK RCNN model. The images are segmented prior to feature extraction; the DenseNet deep model is then cognized of the results. Two distinct layers are utilized in the feature extraction process: a fully connected layer and an average pool. The proposed methods achieve accuracies of 94.8%, 88.5%, and 96.3% on HAM10 0 0 0, ISBI2016, and ISBI2017.

Khan et al. [63] constructed a model using Resnet50 and the feature pyramid network (FPN) as its foundational components. The most optimal CNN features are acquired by employing softmax classifiers for the final classification. To validate the segmentation method’s accuracy, the following three datasets are applied: PH2, ISBI2016, and ISIC2017. The classification task is performed using the HAM10000 dataset based on the selected metrics (86.5% accuracy, 85.57% sensitivity, 87.01% precision, and 86.28% F1 score).

He X et al. [64] implemented a CAFNet in which two branches are used to extract properties from dermoscopy images, and a hyper-branch is employed to augment and integrate these properties at each network level. The hyper-branch is composed specifically of several co-attention fusion (CAF) modules. To enhance the dependability of the prediction outcome, we additionally propose the incorporation of a combination of prediction and deep-supervised loss methodology with a mean accuracy of 76.8% when applied to a dataset consisting of seven-point criteria. Nawaz et al. [65] present fuzzy k-means clustering (FKM), a fully automated deep learning approach, combined with RCNN to facilitate the segmentation of melanoma in its early stages. The proposed method pre-processes the images of the dataset in order to enhance the visibility of the details and reduce noise. The efficacy of the proposed methodology is evaluated through the utilization of three well-established datasets. The described methodology attains an average accuracy of 95.6% when implemented on the PH2 dataset, 93.1% when implemented on the ISIC-2017 dataset, and 95.40% when implemented on the ISIC-2016 dataset.

A fully convolutional encoder–decoder network (FCEDN) was introduced by Mohakud et al. [66] for segmenting dermoscopy images. The achievement of hyper-parameter optimization for networks can be accomplished through the implementation of a distinctive methodology known as Exponential Neighborhood Grey Wolf Optimization (EN-GWO). The assessment is conducted using the datasets associated with 2016 and 2017 that were obtained from the International Skin Imaging Collaboration (ISIC). The effectiveness of the proposed model in partitioning images representing skin cancer is demonstrated with an accuracy of 98.32% on ISIC 2016 and 87.23% on ISIC 2017. Mukadam et al. [67] utilized a CNN to classify the seven distinct classifications that were discovered in the HAM10000 database, and a model based on Convolutional Neural Networks (CNNs) was constructed. The experimental model that has been proposed attains an accuracy of 98.89% to categorize different types of skin cancer. Afza et al. [68] utilized deep learning and two-dimensional superpixels. An initial contrast enhancement is applied to the dermoscopy photos. To obtain feature information, the mapped images are inputted into a transfer learning-trained deep learning model (ResNet-50). After the features have been restored, they are subjected to an enhanced grasshopper optimization procedure prior to being categorized utilizing the Naïve Bayes classifier. Three datasets (containing three, two, and seven skin cancer classifications, respectively) were utilized to assess the proposed hierarchical method. The accuracy ratings achieved through the implementation methodology were 85.50%, 95.40%, and 91.1% on the Ph2, ISBI2016, and HAM1000 datasets.

Additionally, a novel hybrid architecture was suggested by Sayed et al. [69] in which the optimization of bald eagle search (BES) is incorporated with a convolutional neural network. This study proposes a data augmentation method that employs a random oversampling strategy. After evaluation, the proposed model for predicting melanoma skin cancer achieved an overall accuracy of 98.37%. A multi-stage framework for melanoma identification was proposed by Alenezi et al. [70]. In order to show features in dermoscopy photos and eliminate hair details, this model devises a useful pre-processing technique that involves dilatation and pooling layers. The feature extractor for the processed photos was a deep residual neural network. Ultimately, the support vector machine (SVM) classifier received these chosen characteristics as inputs. The ISIC-2019 and ISIC-2020 datasets were utilized to assess the performance of the proposed model. Consequently, the suggested model had a 99% accuracy rate when identifying benign or malignant skin lesions from the image data. Jasil et al. [71] introduced the Densenet and residual network, a novel convolutional neural network (CNN) architecture that utilizes contextual information. The efficacy of the classifier is enhanced by up-sampling the data and adding more information to it. The outcomes of the experiments demonstrate that the method achieves 95% accuracy in the automated categorization of skin lesions using the Ham10000 dataset.

A novel classification network (MFEC net) was proposed by Bindhu et al. [72], which operates on the principle of multi-stage feature extraction. The fuzzy U-net is utilized to segment the lesion, while the PCA is utilized to derive the structure-based features. The color-level co-occurrence matrix is employed to extract color features from the photos, while the gray-level co-occurrence matrix is utilized to retrieve texture features from the photos. The deep belief network is used to differentiate between benign and malignant stages in skin lesion images. The MFEC net technique attains an average accuracy of 98.79%. Tabrizchi et al. [73] presented a novel approach for the early detection of skin cancer based on image processing derived from dermoscopy. The model is constructed by utilizing the widely recognized VGG-16 network. This analysis was conducted using the ISIC dataset. The results indicate that the accuracy of the proposed model is superior to that of the alternative techniques that were assessed.

Alam et al. [74] employed the convolutional deep neural network S2C-DeLeNet. It can classify individual images according to the particular medical condition illustrated in each image, and it segments lesion-based regions from dermoscopy images in comparison to the undamaged skin tissue. The segmentation employs the EfficientNet-B4 as a core network instead of the encoder. On the other hand, the classification sub-network forecasts lesions through the utilization of learned segmentation feature maps and a “Classification Feature Extraction” strategy. Similarly, the classification process attains a mean accuracy of 0.9103. A CNN model was trained using the HAM10000 dataset in order to classify seven unique types of cutaneous lesions. A total of 91.51% of the data were classified with precision by the model. The model underwent two evaluations by seven board-certified dermatologists before being incorporated into an online program. In the initial phase, the model achieved an accurate classification of skin lesions in 90.28% of the cases. In the succeeding phase, the model corrected the mistaken diagnosis made by the specialists by 11.14 percent [59].

An evaluation was conducted on the classification accuracy of the DL and ML models. In terms of accuracy, this work demonstrates that the DL methods outclass the ML algorithms utilized in the proposed study [75]. Dong et al. [76] utilized the CNN model, and the proposed system was able to attain the highest level of accuracy (95.18 percent). To enhance the efficacy of the classification network and reveal latent discriminative features, we propose the implementation of Cross-Modality Collaborative Feature Exploration (CMC). The proposed method is assessed using four publicly available datasets of skin lesions: ISIC 2018 and PH2 for segmentation purposes and ISIC 2019&2020 for classification. The method achieves an accuracy of 92.63% in skin lesion classification.

Qureshi et al. [77] presented a novel convolutional neural network (CNN) architecture that integrates multiple CNN models via a meta-learner. The benefits of the proposed methodology are demonstrated through the utilization of a dataset consisting of 33,126 dermoscopic images of 2056 individuals. Panthakkan et al. [78] integrated Xception and ResNet50 to categorize the seven different lesions present in the HAM10000 dataset. The sliding window method was utilized in the training and evaluation of the models. The most sophisticated model is the concatenated X-R50, which achieves a prediction accuracy of 97.8%.

The SCSO-ResNet50 technique was introduced by Akilandasowmya et al. [79] to ensure accurate forecasts by identifying deep hidden features. The enhanced harmony search (EHS) method is used to reduce the complexity of the data and maximize their characteristics. Ensemble classifiers, including Naive Bayes, KNN, SVM, linear regression, and random forest, are utilized for the early identification of cancer. The efficacy of the proposed method is evaluated using two datasets: the ISIC 2019 dataset and the Kaggle skin cancer dataset. Table 1

An overview of current studies on the use of several DL and ML models for skin cancer diagnosis.

Ref.	Year	Models	Dataset	Disease Classification	Accuracy
[58]	2021	CNN	HAM10000	Multiclassification	91.93%
[59]	2021	DCNN	HAM10000	Multiclassification	91.51%
[62]	2021	MASK RCNN	HAM10000	Multiclassification	94.80%
[63]	2021	PAM-DenseNet	HAM10000	Multiclassification	86.50%
[64]	2023	CAFNet-34	Seven-Point Checklist	Multiclassification	76.80%
[65]	2022	RCNN	ISIC 2016 ISIC 2017 PH2	Binary Classification	95.40% 93.10%95.60
[66]	2022	FCEDN	ISIC 2016 ISIC 2017	Binary Classification	98.32% 87.23%
[68]	2022	Superpixal DL	ISIC 2016 HAM10000 PH2	Binary Classification	95.40% 91.10 85.50
[69]	2021	BES NN	ISIC 2020	Binary Classification	98.37%
[72]	2023	Spiking VGG-13	ISIC 2019	Binary Classification	89.57%
[73]	2023	VGG 16	ISIC Archive	Binary Classification	86.30%
[74]	2023	S2C-DeLeNet	HAM10000	Multiclassification	91.03%
[79]	2024	SCSO-ResNet50	ISIC 2019	Multiclassification	93.45%

Multiple studies [58,59,60,61,62,63] have shown that many types of skin cancer, such as AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn, have comparable sizes and shapes. Medical practitioners find it difficult to accurately diagnose melanoma and other skin lesions based on dermoscopy photos. An automated system is therefore required to automatically diagnose the aforementioned skin cancer types using dermoscopy photos. The main goal of previous studies [27,46,47,48,49,50,51,52,53,54] was to distinguish melanoma from non-melanoma cases using dermoscopy images. Multiple researches based on deep learning use dermoscopy images to detect various types of skin malignancies [13,14,15,16,41,42,43,44,45,46,47]. The task of automatically classifying skin cancer in dermoscopic images is difficult because of the high levels of visual similarity and intraclass variation. Furthermore, it is extremely difficult to classify skin cancers into different categories due to the existence of both intrinsic and extrinsic artifacts, as well as the differentiation between skin that is normal and skin that is affected. Consequently, this study aims to develop a framework based on DL and HC that can detect and classify various types of skin malignancies using dermoscopy images. As a result, researchers will be able to overcome the previously mentioned challenges.

3. Dataset Description

3.1. ISIC 2019 Skin Cancer Dataset

The renowned dataset from ISIC 2019 consists of 25,331 images comprising eight distinct types of skin cancer: AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn. This dataset includes the images of the MSK Dataset, HAM10000 dataset, and BCN_20000 dataset, and the images of this dataset were obtained in a JPEG format [80]. In this study, we used 450 AKs images, 650 BCCa images, 850 BK images, 250 DFa images, 850 MELa images, 1550 MNi images, 250 SCCa images, and 250 VASn images, and Figure 1 shows the images of skin cancer.

3.2. Handling Imbalanced Class Dataset

ISIC 2019 is an unequal dataset; there is a significant number of occurrences in one class and only a few occurrences in the other classes. Instances belonging to minority groups are incorrectly classified as a consequence of the unequal distribution of classes, making the classifier system biased and inclined towards instances that belong to the majority [81]. In the ISIC 2019 dermoscopy image databases, most categories of skin cancer are imbalanced, as Table 2 shows. As a result, we use SMOTE Tomek to raise the percentage of photos associated with minority-class diseases across all classes in the dataset. Table 3 compiles all dermoscopy photos associated with skin cancer after the SMOTE Tomek procedure.

4. Proposed Methodology

The proposed methodology for classifying dermoscopic images into several classes is discussed in this section. The whole process of the suggested method is shown in Figure 2. The following sections illustrate the several procedures involved in classifying dermoscopic images, as seen in Figure 2.

4.1. Pre-Processing

Hairs on a dermoscopic image can conceal the shape, texture, and form of a skin lesion. The presence of hair significantly enhances the difficulty in acquiring features. In this study, we provide a method for hair removal in dermoscopic images. Moreover, image normalization and image resizing are performed at this stage.

Hair Removal Process

The RGB dermoscopic images are converted to grayscale to complete this operation. A black-hat (BH) transformation uses a structural characteristic with a cross shape of 15 × 15 to identify thin-stranded hairs. The BH transform is used to locate the grayscale image’s intensity. This method computes the difference between the pre-morphological closure photo and the post-morphological closure photo [82]. In the next step, binary thresholding is applied to the images to exclude pixels that represent hairs as white and have values less than 20. The hair reduces the accuracy of the classification; therefore, a hair removal mask conceals the margins and texture of the skin lesion if the threshold value is lowered. Binary thresholding is a portraying mask that creates a binary image composed of hair strand pixels. The potential pixels are painted in the pixels around them to provide the impression that they are in a neighborhood [83]. The BH transformation results are shown in Figure 3.

4.2. Feature Extraction

A set of feature descriptors is needed that helps the classifier distinguish between dermoscopic images belonging to various categories. The proposed SNC_Net and B1, B2, B3, and B4 extract features from pre-process dermoscopic images.

Several studies conclude that HC features are simple to extract, particularly from the small size of data sets. To decide which attributes should be eliminated, experts in the relevant field are contacted. Compared to other aspects, HC attributes are simpler to observe and do not need any kind of training set. However, there exist instances when it becomes challenging to distinguish between the features in complex images; in these cases, DL methods can be used for feature extraction. These DL models are capable of automatically extracting features from an input image, but to derive features of good quality, they need a large and highly diverse training sample. Moreover, low-level features must be extracted, which leads to a more thorough description of the image and facilitates skin cancer detection and classification. The proposed method extracts features by using both the HC and DL approaches. Because the merged feature generates a higher-quality feature vector, it results in better classifier performance. Since a segmentation mask that removes all background data sometimes causes a rapid decline in the classifier’s performance, segmentation is not performed in this study. The tissue around the skin lesion has valuable information that is lost when the tissue is removed in segmentation, which lowers the classification accuracy [84,85].

4.2.1. Feature Extraction Using Handcrafted Method

The collection of useful HC features facilitates the process of classifying skin lesions. These characteristics include shape, color, local, global, and textural feature elements. We set up histograms to extract color characteristics in five distinct color spaces: grayscale, RGB, HSV, YCrCb, and L*a*b. To differentiate the different classes of skin lesions, color characteristics are crucial. The histograms are used to calculate features like kurtosis, mean, skewness, and standard deviation. All color spaces (except grayscale) contain 52 color characteristics, each with three color components. Dermoscopic images are analyzed using the GLCM to extract global textural information [86]. The values of 0°, 45°, 90°, and 135° are the four GLCM production orientations that add up to 52 global textural features. FAST and Rotated BRIEF are the two methods that are used to extract local textural information. Combining the FAST feature detector with the BRIEF feature descriptor integrates ORB. It uses FAST to identify important regions of the images. We found the 64 most important key points by employing the Harris corner method, in which each key point indicates a distinct feature [87]. One of the most widely used methods for figuring out an image’s shape features is the Zernike moments approach. This shape feature descriptor is particularly helpful in extracting features from images with complicated borders. Zernike moments are resistant to noise and represent images with fewer details. The proposed technique describes the skin lesion using the eight orders of Zernike moments that are taken out of each dermoscopic image [88].

4.2.2. Feature Extraction Using Inception V3

The distinct layers that make up a convolutional neural network (CNN) are the input, pooling, convolution, fully connected, and classification layers. Inception V3 is the network that is built by Google. It uses the Inception model to connect the layer attributes and improve the depth [89]. When certain pixels have unique parameters and biases relative to the preceding layers, adjustments are made to the convolutional layers. The parameters and biases are applied to the image after it is divided into smaller sections. These parameters and biases are referred to as filters. To generate feature maps, these filters are combined with the identified small regions in the input image. The filters are used to determine the specific attributes from the image that provide data to the input layer. Convolutional operations also utilize multiple parameters and engage a single feature to analyze an entire image through the application of a singular filter. The convolutional layer hyper-parameters consist of its stride, number of filters, buffering, and local area size. In order to achieve the most favorable outcomes, the hyperparameters are adjusted to match the dimensions of the input image. Layer of Pooling (PL) reduces the geographic range, number of components, and complexity of an image. The constant technique is used when dealing with an input that has no parameters. A wide range of PL variants are accessible, such as stochastic, average, and max pooling. The most prevalent variant is max pooling, which reduces the input when n × n is applied. In situations where the input capacity is constrained, the n × n region is utilized to its maximum potential. Translational consistency is attained when an input image is assessed with a little change in location. The place therefore becomes smaller and eventually vanishes. PL’s output serves as an input for the fully connected layer. The network could operate like a convolutional neural network since every neuron is linked to the current layer. Consequently, the convolutional layer has the greatest parameters out of all the layers. The last layer, known as the classification layer, is linked to the fully linked layer. Various CNN versions use distinct activation functions (AF). Compared to tangent and sigmoid functions, non-linear activation functions provide superior results. The purpose of these functions is to speed up the training process. The chain rule and vector computation are used by convolutional neural networks (CNNs). Assume that I is a scalar as ieR and j is a vector as jRh. i is a function of j and denotes the partial derivative. Its mathematical expression is as follows:

(1)$\left(\genfrac{}{}{0pt}{}{\partial i}{\partial j}\right)=\frac{\partial i}{\partial j_n}$

In particular, $\left(\genfrac{}{}{0pt}{}{\partial i}{\partial j}\right)$ denotes the vector whose magnitude is equal to j, and the nth the vector denoting the nth component of the number is denoted by $\left(\genfrac{}{}{0pt}{}{\partial i}{\partial j}\right)$n. In particular, $\left(\genfrac{}{}{0pt}{}{\partial i}{\partial j^s}\right)={\left(\genfrac{}{}{0pt}{}{\partial i}{\partial j}\right)}^s$.

Furthermore, j is a function of k, whereas k eR^w represents a singular vector. Additionally, this is the fractional derivative of i stated in terms of k:

(2)${\left(\genfrac{}{}{0pt}{}{\partial i}{{\partial j}^s}\right)}_{xy}=\frac{\partial i}{\partial k_x}$

It is found that the fractional derivative is the d x e matrix, as in $(\partial i)/(\partial k)$ at the intermission of the x and y, respectively. Chain arguments make it obvious that z owes x anything. Because of this, some methods map k to i, and others map j to i. The calculation that is seen below was completed using the chain approach.

(3)$\left(\genfrac{}{}{0pt}{}{\partial i}{{\partial k}^s}\right) as \left(\genfrac{}{}{0pt}{}{\partial i}{{\partial k}^s}\right)=\left(\genfrac{}{}{0pt}{}{\partial i}{{\partial j}^s}\right) \left(\genfrac{}{}{0pt}{}{\partial i}{{\partial k}^s}\right)$

The loss computation is employed to determine how much the actual value differs from the expected value of the CNN i¹ and the goal; $k^1\to e^1, e^2\to ,\dots ,k^1\to e^1=i$. The loss function may be easily understood: i = ||target − k^a||². The expected output is shown as argmax_x $k_x^a$. Consequently, a convolutional operation may be computed as follows:

(4)$j_x^{a+1}, x^{a+1},f=\sum _{x=0}^d\sum _{y=0}^e\sum _{f=0}^{F}hxyf\times k_{x^{a+1}}^a+x,y^{a+1}, f$

where the symbol f stands for the filter of size (d x e x f¹). The size (d¹ − d + 1) x (e¹ − e + 1) is therefore preserved by the conv. layer, which is made up of f slices that represent ${\mathrm{j}(k}^{a+1}) in R^{d^{a+1 x a+1 }},d^{a+1}=d^a-d+1, e^{a=1}=e^a-e+1 and f^{a+1}=f$ and is used to determine the probability of all labels l e{1, …, l} for the training instance.

(5)$a=\sum _{l=0}^m\mathrm{l}\mathrm{o}\mathrm{g}\left(q\right(l\left)\right)t\left(l\right)$

The cross-entropy is computed using differentiation when gradient training is applied to deep functions and it has the simplest form, q(l) t(l), which ranges from −1 to 1. The cross-entropy decreases when there is a chance that the probability of a right label will reach a maximum value. When the labels in Inception version 3 lack training instances, they are said to be mutual (v(l)); the shared label s(l|k) is hence

(6)$p^{\prime \left(t\right|k)}=\left(1-\in \right){\delta }_{l,k}+\frac{\in }{L}$

In an alternative scenario, cross-entropy is calculated as follows:

(7)$g\left(p^{\prime },q\right)=\sum _{l=0}^m\log \left(q\left(l\right)\right)p^{\prime \left(l\right)}=\left(1-\in \right)g{(r}^{\prime },q)+\in g(w,q)$

Thus, for losses represented by g(p,q) and g(v,q), the normalization for label smoothing and the cross-entropy loss g(p,q) are the same. Three different-sized max-pooling and convolutional layers make up the Inception network [43]. A variety of channels traverse the network layers subsequent to convolutional operation, and then the non-linear fusion approach is used. The key design of the Inception network is demonstrated in Figure 4. ImageNet serves as the pre-training dataset for Keras’s third iteration of the Inception network. The input images are resized to 299 × 299 for Inception V3. Conversely, Inception V3 divides integrals into the smallest convolution using a convolutional kernel technique; 3 × 3 convolutions may always be divided into 1 × 3 and 3 × 1 convolutions. Spatial feature extraction improves the network speed and is efficient since it needs minimal features. The sizes of the grids are 8 by 8, 17 by 17, and 35 by 35. Figure 4 depicts the basic layout of the Inception model [90,91].

4.3. Feature Fusion Process

Feature fusion is used in numerous machine learning and computer vision contexts, including medical imaging [49,50]. It provides a crucial method for merging the vast majority of feature maps. The entropy-based approach to integrating attributes is proposed. Moreover, a single vector is created by combining the acquired features. Three vectors were computed as follows:

(8)${\mathcal{F}}_{InV3\times i}=\{{InV3}_{1\times 1},{InV3}_{1\times 2},{InV3}_{1\times k} \}$

(9)${\mathcal{F}}_{HC\times j}=\{{HC}_{1\times 1},{HC}_{1\times 2},{HC}_{1\times k} \}$

At this point, $\mathcal{F}$ represents a fused feature vector. The result is then used to calculate an entropy for a few attributes, as shown below.

(10)$Feature {Fusion}_{vector(1\times n)}=\sum _{l=0}^m\left\{{InV3}_{1\times i},{HC}_{1\times j} \right\}$

(11)$N_{pe}={-Mpe}_a\sum _{i=1}^{k}q\left({fe}_i\right)$

(12)${Fe}_S={-N}_{pe}\left(\max \left({fe}_i\right)\right)$

Here, $N_{pe}$ is an entropy and q is the probability of features. Eventually, the affected photos are identified by putting the selected features into the classification network.

4.4. Classification Using CNN

CNNs are one of the most widely used techniques, which use feature vectors to perform mathematical linear operations [92]. During training, a CNN functions in two stages, namely the propagation phases, which are forward and backward. The filter matrix multiplies the input and weights then a convolutional operation determines the output. This output is used to calculate errors that occurred during the forwarding step. The settings are adjusted to take into consideration the final prediction errors during the backpropagation process. One way to find errors is to compare the result with the ground truth and use the cost function [93]. In order to decrease error, the parameter’s gradient is calculated, and thereafter, the parameters are adjusted. A CNN performs well for the classification task when the dataset is image-based [90]. In this study, CNN is utilized to multiclassify skin cancer into eight distinct categories.

4.5. Baseline Models

4.5.1. EfficientNetB0 (B1)

To achieve optimized performance, the Efficient Net architecture balances the model’s depth, breadth, and resolution. The primary idea is to achieve an ideal balance between the model size and accuracy. EfficientNetB0 is utilized when minimal computational resources are required. EfficientNetB0 is widely recognized for its precise performance and minimal computational requirements in addition to its rational layer count. It is widely used in several computer vision applications, including image classification and object detection [30].

4.5.2. MobileNetV2 (B2)

MobileNetV2 is a convolutional neural network architecture that has been expressly developed for devices with limited computational resources. It is an improved version of the original MobileNet architecture. It employs depth-wise separable convolutions, inverted residuals, and linear constraints to enhance its efficacy. In scenarios where computational resources are limited, its performance in object detection and image classification is exceptionally strong [31].

4.5.3. DenseNet-121 (B3)

DenseNet-121 is a variant of the densely connected convolutional network DenseNet. The network’s name incorporates the number “121” to represent the number of layers. Each preceding layer of DenseNet-121 provides a direct input to the subsequent layer, complying with the dense connectivity pattern. This connectivity pattern improves the overall efficacy of information transmission across the network through the utilization of features. The efficacy of DenseNet-121 in training deep neural networks and its capacity to address the issue of vanishing gradients are widely acknowledged [32].

4.5.4. ResNet-101 (B4)

ResNet-101 is the variant of ResNet. The number “101” in the name represents the number of network layers. The ResNet architecture is extensively acknowledged for its incorporation of residual blocks, which skip connections to facilitate the acquisition of residual mappings by the network. This architectural design effectively addresses the challenge of training exceptionally deep neural networks by incorporating an adaptive solution for the vanishing gradient problem. ResNet-101 has been widely deployed in the domain of computer vision to perform an extensive range of functions, including image classification and object detection. The trade-off between the computing efficiency and model depth makes it suitable for a broad range of applications [33].

4.6. Performance Evaluation

We analyzed the dermoscopy images to examine how well the SNC_Net performs to classify the eight types of skin cancer. After every model was trained, the data from every stage of the suggested method were used to create the confusion matrix-based performance parameters. A range of metrics, including accuracy (Accu), recall (Rec), F1 score (FS), precision (Pre), true positive rate (T_p), true negative rate (T_N), false positive rate (F_p), and false negative rate (F_N), were used to assess the identification performance of the SNC_Net models on the testing dataset. The parameters may be measured using Formulas (13)–(17).

(13)$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}=\frac{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{T}}_{\mathrm{N}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{P}}+{\mathrm{T}}_{\mathrm{N}}+{\mathrm{F}}_{\mathrm{N}}}$

(14)$\mathrm{S}\mathrm{e}\mathrm{n}=\frac{{\mathrm{T}}_{\mathrm{P}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{N}}}$

(15)$\mathrm{P}\mathrm{r}\mathrm{e}=\frac{{\mathrm{T}}_{\mathrm{P}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{p}}}$

(16)$\mathrm{R}\mathrm{e}\mathrm{c}=\frac{{\mathrm{T}}_{\mathrm{P}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{p}}}$

(17)$\mathrm{F}\mathrm{S}=\left(\frac{\mathrm{P}\ast \mathrm{R}}{\mathrm{P}+\mathrm{R}}\right)\ast 2$

5. Experimental Results

In this section, we compare the most recent deep network with SNC_Net. Four different basic deep networks are analyzed and evaluated in Table 2 using the proposed SNC_Net. The same set of parameters are used to assess the effectiveness of each deep neural network.

5.1. Experimental Setup

Four baseline models and the proposed SNC_Net model were among the five models that were successfully constructed using Keras. Python was used as a programming language; 32 gigabytes of RAM and a 12 GB NVIDIA GPU were used in a Windows 10 PC for the experiment.

5.2. Accuracy Compared with Other Models

We compared our proposed SNC_Net with four existing baseline networks, B1, B2, B3, and B4, using SMOTE Tomek. Furthermore, we compared the proposed SNC_Net before implementing the SMOTE Tomek. Up-sampling is used by the system to provide outstanding results for the recommended model. Table 4 displays the accuracy values for the proposed SNC_Net without SMOTE, SNC_Net with SMOTE, B1, B2, B3, and B4, which are 91.45%, 97.81%, 93.39%, 95.21%, 92.68%, and 95.80%. Figure 5 illustrates the significant improvement that is possible with the proposed SNC_Net with SMOTE.

5.3. AUC Comparison of Recent Deep Models with Proposed SNC_Net

Our suggested model, SNC_Net, is built on HC and DL methods and features several units that are very successful in classifying various types of skin cancer. We evaluated our proposed SNC_Net by comparing it with four baseline networks. The following AUC values were reached by the baseline networks B1, B2, B3, and B4: 99.14%, 99.21%, 98.99%, and 99.43%. Using the same dataset, Figure 6 shows that the AUC values of 97.51% and 99.67% were obtained by the SNC_Net without SMOTE and SNC_Net with SMOTE. The SNC_Net AUC values are greater than those of the baseline models.

5.4. Comparison of Proposed SNC_Net with Other Networks Using Precision

We compared SNC_Net using SMOTE Tomek with B1, B2, B3, and B4 on the same dataset. The proposed SNC_Net using the SMOTE method yielded amazing results. The suggested SNC_Net with and without SMOTE earned precision values of 98.31% and 91.82%, respectively, whereas B1, B2, B3, and B4 achieved 93.52%, 95.58%, 91.99%, and 95.79% precision, respectively. Our research indicates that the proposed SNC_Net with SMOTE performs better than the baseline networks, as shown in Figure 7.

5.5. Comparison of Proposed SNC_Net with Other Networks Using Recall

The method involves dividing the overall number of true accurate positives by the total number of accurate positive predictions. The model needed to identify positive samples is assessed using the recall metric. Greater recall percentages indicate an abundance of accessible positive samples. As shown in Figure 8, recall curves are utilized to compare the proposed SNC_Net to the baseline networks. The recall percentages were 97.89%, 91.70%, 93.15%, 95.23%, 92.55%, and 95.44% for the proposed SNC_Net with and without SMOTE, B1, B2, B3, and B4, respectively. The previously given complete explanation may be the reason for the proposed strategy’s remarkable recall effectiveness.

5.6. F1 Score Compared of Proposed SNC_Net with Recent Models

The suggested methods, SNC_Net without SMOTE and SNC_Net with SMOTE, produced F1 score values of 91.70% and 98.10%, respectively. Four baseline networks—B1, B2, B3, and B4—have F1 scores of 93.49%, 95.33%, 92.24%, and 95.85%, as shown in Figure 9.

5.7. Loss Comparison of Proposed SNC_Net with Other Deep Networks

The difference between the expected and actual numbers is calculated using loss functions. The categorical cross-entropy method was used to compute the research’s loss. When up-sampled photos were used to build the network, the findings became much more remarkable. The suggested SNC_Net produced loss values of 0.0512 and 0.8417 with and without SMOTE, while B1, B2, B3, and B4 obtained loss values of 0.2516, 0.2244, 0.2056, and 0.2124, respectively. This significant reduction in SNC_Net loss with SMOTE is seen in Figure 10.

5.8. ROC Comparison of Proposed SNC_Net with Other Deep Networks

ROC is used to evaluate the precision of disease diagnoses in connection to classifier prediction. The performance of a classifier is assessed by examining the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) curve; a greater AUC signifies a more effective classifier. We evaluated the accuracy of our proposed SNC_Net on the curve with and without SMOTE using the dataset. Using the same dataset, this curve contrasted the suggested SNC_Net with and without SMOTE to the four deep models. The proposed SNC_Net with and without SMOTE is illustrated in Figure 11. The ROC values for B1, B2, B3, B4, and SNC_Net without and with SMOTE are as follows: 0.9023, 0.9341, 0.9256, 0.9413, 0.9257, and 0.9736, respectively. Figure 11 demonstrates how up-sampling the proposed SNC_Net significantly enhances the ROC curve.

5.9. Values of AUC (ROC) Extension Compared to Other Models

An enhancement of the ROC curve is utilized, as shown in Figure 12, to depict a comparison between the SNC_Net and four baseline networks. Following the implementation of the SMOTE technique to achieve equilibrium in the dataset, the performance of the proposed method demonstrated a substantial enhancement in comparison to the four models illustrated in Figure 12. The AUC values for the classes supplied by the proposed SNC_Net—class 0 (AKs), class 1 (BCCa), class 2 (SCCa), class 3 (BKs), class 4 (DFa), class 5 (MNi), class 6 (MELa), and class 7 (VASn)—both with and without the implementation of the SMOTE technique also demonstrated a substantial effect. The observable improvements in AUC provide support for the effectiveness and reliability of the feature selection process and the SMOTE technique in SNC_Net.

5.10. Confusion Matrix Comparison of Proposed SNC_Net with Other Networks

To evaluate our proposed model using the confusion matrix, we examined it with other networks. As seen in Figure 13, the system applied to SMOTE results in effective growth for SNC_Net. Although the SNC_Net demonstrated accurate classification for 121 out of the 134 images presented in the AKs instances, it made errors by misidentifying 14 images. Based on the data depicted in Figure 13, 135 out of a possible 147 BCCa images were accurately classified as BCCa, while 13 images were misclassified as other diseases in error. The SNC_Net incorrectly classified 27 images for BKs and it accurately identified 111 out of 138 total output images as BKs. SNC_Net accurately identified 136 DFa images out of a total of 139 images. In standard operating conditions, 129 images were categorized as MELa, while 20 images were categorized to the wrong class. The precise detection of 100 MNi images out of a total of 130 images was accomplished by the proposed method. A total of 123 out of 125 images were correctly identified as SCCa by SNC_Net, whereas 119 images of VASn were correctly identified, and only 2 images were wrongly classified.

Furthermore, we employed Grad-CAM heatmaps to visually depict the results generated using our proposed model. The heatmap that is illustrated in Figure 14 serves the purpose of graphically depicting the particular region of interest to which the model is allocating its resources.

5.11. Ablation Study

An ablation study was performed on several parameters. In the first experimentation evaluation, four layers, namely Average Pooling 2D, Max Pooling 2D, Dropout, and Flatten, were used. The efficiency of the proposed model was observed by changing these layers. The results show that the AveragePooling2D layer achieved the same performance, whereas when MaxPooling2D was used, the performance declined. Similarly, the performance remained the same when the dropout layer was used. In the second experiment, the batch size changed. It was observed that the proposed model gives maximum performance on a batch size of 8, whereas the performance decreases when the batch size is increased. In the third experiment, the optimizers were changed. The proposed model achieved maximum performance when the Adam optimizer was used, and its performance decreased with the Nadam and Adagard optimizers. The best results were achieved by the proposed model when the learning rate was set to 0.0001. When the learning rates were increased, the performance of the model decreased. Table 5 provides a brief summary of the ablation study.

Moreover, ablation studies are frequently performed for CNN-based applications in order to evaluate the efficacy and stability of the model. These investigations require the elimination or adjustment of a variety of layers and hyperparameters. The network’s performance could change in a number of ways, from improving to worsening, based on the modifications made to the model’s components. To improve accuracy, it is common practice to conduct experiments using multiple hyper-parameters, such as an optimizer, learning rate, layers, and batch size. Altering the architectural design of the model has the potential to impact its overall efficacy. The aim of this study is to investigate diverse configurations of the proposed model by randomly removing or modifying particular components and attributes. Three case studies are carried out, and the outcomes are analyzed. The findings indicate (see Table 5) that the utilized methodology was successful within the scope of this investigation, as evidenced by the overall improvement in accuracy.

5.12. A Comparison of the SNC_Net Model with the State of the Art

Table 6 displays the outcomes of a comparative analysis between the proposed SNC_Net model and other methods regarded as the state of the art. The table provides comprehensive information concerning the F1 score, recall, precision, and accuracy of each specific method.

5.13. Discussions

By employing images captured by dermoscopy [15,16,17,18,38,39,40,41,99,100], a wide range of skin lesions can be identified and categorized. Through the utilization of a methodology that facilitates an exhaustive examination of a specific location, it is possible to discern both the illness and internal components that have been infected. Dermoscopy is the most precise, efficient, and accurate method for distinguishing between AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn diseases [36,37,38,39,40]. In light of the escalating prevalence of confirmed skin cancer cases, a computerized diagnostic approach is necessary for the identification of AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn [56]. By employing methodologies derived from the domain of DL [92,93], dermoscopy images have the capacity to autonomously distinguish between various lung diseases and those associated with other ailments. As a result, we constructed the SNC_Net model, which is constructed upon DL and demonstrates the capacity to precisely diagnose an extensive spectrum of pulmonary disorders. By utilizing this model, radiologists are able to facilitate the initiation of treatment for patients suffering from various conditions, including AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn. The evaluation of the proposed SNC_Net model’s efficacy was conducted using the ISIC 2019 benchmark dataset, which is available to the public [70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94]. In conjunction with the outcomes of several baseline models—B1, B2, B3, B4—the suggested SNC_Net efficacy was evaluated. The image derived from the datasets exhibits a noticeable imbalance, which is elaborated upon in Table 2. The model’s training efficacy is compromised due to the inequitable class distribution observed across the images [74,95]. In order to address these challenges, we utilized the SMOTE Tomek method to enhance the images by including individuals from underrepresented minority groups in the databases [67]. Our proposed SNC_Net model has been adequately trained on the eight categories of skin cancer (AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn), as depicted in Figure 6. As a result, it is capable of accurately classifying instances of infection associated with each of these subcategories. The superior performance of our SNC_Net model in classifying thoracic diseases compared to the other four baseline classifiers for skin cancer diseases is evident in Table 4. With respect to the classification of dermoscopy images into AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn, the SNC_Net model, which is proposed and utilizes the SMOTE Tomek method, achieved a precision of 97.56%. Furthermore, the SNC_Net model achieved an accuracy of 91.82% when SMOTE Tomek was absent. On the other hand, the B1 architecture exhibited a commendable accuracy rate of 93.39%. The accuracy ratings of the B2 and B3 models were identical at 95.58% and 95.21%, respectively. The efficacy of the B4 in classifying skin cancer was comparatively lower than that of all baseline models. Figure 14 also shows the GRAD-CAM assessment of the suggested SNC_Net model for the categorization of skin cancer.

Abunadi et al. [10] utilized deep learning and two-dimensional superpixels. An initial contrast enhancement was applied to the dermoscopy photos. The hybrid characteristics that were obtained via LBP, GLCM, and DWT served as the foundation for the proposed system. Following their integration into a feature vector, these features were categorized via the use of FFNN and ANN classifiers. On the ISIC 2018 dataset and the PH2 dataset, the FFNN approach achieved diagnostic accuracy values of 95.24 percent and 97.91 percent, respectively. Second, the AlexNet and ResNet-50 models were used in the transfer learning strategy to diagnose skin disorders. In terms of patient diagnosis, the ResNet-50 model showed an accuracy rate of 90% on the ISIC 2018 dataset and a 95.8% accuracy rate on the PH2 dataset. Qasim et al. [94] used deep spiking neural networks along with the surrogate gradient descent method to perform binary classification on the images collected from the ISIC 2019 dataset. The spiking VGG-13 model achieved an accuracy of 89.57%. Tembhurne al. [95] presented a novel approach that integrates machine learning and deep learning techniques to address the difficulty associated with detecting skin cancer. The machine learning model assesses features produced by employing techniques like the LBP and HCT. In contrast, the deep learning model utilizes state-of-the-art neural networks to extract properties from images. In the event of a problem arising during image categorization, it is necessary to address it by extracting features. By incorporating both ML and DL features, the suggested approach achieved an improved accuracy rate of 93%. The model was evaluated through benchmarking using the publicly ISIC archive dataset. In order to categorize skin lesions, a deepRNN transformed via wavelet operations was used. The proposed method effectively refines images of skin lesions by separating more minute details and eliminating extraneous features through the utilization of wavelet processing, aggregation, and normalization. The proposed model achieved 95.71 percent accuracy on the ISIC 2017 dataset and achieved 95.84% accuracy on the HAM10000 dataset [96]. Shorfuzzaman et al. [97] presented a stacked ensemble architecture based on explainable CNNs, which is intended to facilitate the early detection of melanoma skin cancer. The final predictions are generated by a meta-learner, which is a new model that incorporates each of the predictions from the sub-models. An open-access dataset comprising images of both benign and malignant melanoma is employed for the purpose of evaluating the model. As indicated by the evaluation results, our ensemble model demonstrates applicability on account of its remarkable sensitivity (96.67%), accuracy (95.76%), and area under the curve (0.957).

SNC_Net consistently provided the greatest results when it came to a number of evaluation parameters, including accuracy, precision, recall, and F1 score. Table 6 illustrates how the enhanced feature extraction technique and creative pre-processing techniques enhance the performance of the SNC_Net. Our model outperforms earlier approaches, not because it has a more intricate learning model but rather because the feature extraction and final classification are carried out more skillfully. Combining DL and HC data during training allowed the models to acquire deep knowledge about the relationships between input features and output labels. Key features are extracted from dermoscopy images using the HC and DL techniques by the SNC_Net, which then combines them into a feature vector that the CNN employs for classification.

6. Limitations of Existing Research

The evaluation of the SNC_Net was performed on the ISIC 2019 dataset, which is a high-class imbalance dataset. We employed SMOTE TOMEK to increase the number of images to balance the dataset. However, a more extensive dataset is needed to provide more accurate results when evaluating the suggested model. There are frequent inconsistencies that occur in the publicly available datasets. To ensure a thorough evaluation of the capabilities of the proposed method, it is necessary to utilize a dataset derived from the real world.

7. Conclusions

In the present study, the SNC_Net model is presented which classify eight distinct kinds of skin lesions (AKs, BCCa, SCCa, BKs, DFa, MNi, MELa, and VASn). Currently, skin cancer is becoming more common and harming individuals all over the world. A significant number of lives have been lost as a consequence of poor facilities, inaccuracies, and slowness in testing methods, as well as the absence of early identification for skin cancer. Due to the significant number of skin cancer instances, a quick and effective testing approach is needed. The SNC_Net model is proposed with the goal of classifying eight different kinds of skin cancers. The CNN is used to categorize illnesses, whereas HC and inception v3 (DL approach) are used to extract significant features from dermoscopy images. The SMOTE Tomek approach is used for sample generation that accomplish the sample balance for each class and resolve the issues related to unbalanced datasets. Grad-CAM offers a heat map of class activation to show how the suggested approach works graphically. The following performance metrics were attained by the suggested SNC_Net: 97.81%% accuracy, 98.31% precision, 97.89% recall, and a 98.10% F1 score. Therefore, it can be concluded that SNC_Net has the potential to serve as a valuable resource for medical professionals. This study’s inability to apply the suggested SNC_Net model to camera-captured pictures is one of its limitations. To increase the accuracy of skin cancer classification, we need to include federated learning.

Author Contributions

Conceptualization, A.N. and T.A.; methodology, A.N., T.A. and K.Z.; validation, R.A.N., T.A., S.-W.L. and M.K.; formal analysis, A.N. and S.-W.L.; investigation, A.N. and R.A.N.; resources, A.N., T.A. and K.Z.; data curation, R.A.N.; writing—original draft preparation, A.N., M.K. and K.Z.; writing—review and editing, A.N. and M.K.; visualization, A.N. and K.Z.; supervision, K.Z., R.A.N. and S.-W.L.; funding acquisition, S.-W.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

Authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. ISIC 2019 images of different classes.

Figure 2. Proposed methodology of SNC_Net for skin cancer classification.

Figure 3. Images after hair and artifact removal using BH transformation.

Figure 4. Inception v3 feature extraction process.

Figure 5. Accuracy comparison of the proposed SNC_Net with four baseline networks: (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 6. AUC values of proposed method with four baseline networks. (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 7. Precision outcomes of proposed method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 8. Recall results of proposed method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 9. F1 score computed between method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 10. Evaluation of values of loss between proposed method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 11. ROC curve comparison of performance of baseline models with proposed method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 12. AUC (ROC) curve extension for other models with proposed method with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 13. Application of confusion matrix to evaluate proposed SNC_Net with other network methods with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 13. Application of confusion matrix to evaluate proposed SNC_Net with other network methods with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 13. Application of confusion matrix to evaluate proposed SNC_Net with other network methods with four baseline networks; (a) B1, (b) B2, (c) B3, (d) B4, (e) SNC_Net without SMOTE, and (f) SNC_Net with SMOTE.

Figure 14. Grad-CAM analysis of proposed SNC_Net for skin cancer detection.

Table 2

Selected samples from ISIC 2019 before SMOTE.

Class Name	Selected Images
AKs	450
BCCa	650
BK	850
DFa	250
MELa	850
MNi	1550
SCCa	250
VASn	250

Table 3

Image samples from ISIC 2019 after SMOTE.

Class Name	Selected Images
AKs	1600
BCCa	1600
BK	1600
DFa	1600
MELa	1600
MNi	1600
SCCa	1600
VASn	1600

Table 4

Performance comparison of SNC_Net with baseline models.

Classifiers	Accuracy	Precision	Recall	F1 Score	AUC
B1	93.39%	93.52%	93.15%	93.49%	99.14%
B2	95.21%	95.58%	95.23%	95.33%	99.21%
B3	92.68%	91.99%	92.55%	92.24%	98.99%
B4	95.80%	95.79%	95.44%	95.85%	99.43%
Proposed SNC_Net(Without SMOTE Tomek)	91.45%	91.82%	91.67%	91.70%	97.51%
Proposed SNC_Net (With SMOTE Tomek)	97.81%	98.31%	97.89%	98.10%	99.67%

Table 5

Summary of experiments for ablation study.

Exp	Layer				Batch Size			Optimizer			Learning Rate			Results
1	AveragePooling2D	MaxPooling2D	Dropout	Flat-ten	8	16	32	Adam	Adagard	Nadam	0.0001	0.00001	0.000001
	✓	-	-	-	✓	-	-	✓	-	-	✓	-	-	Same Performance
	-	-	✓	-	✓	-	-	✓	-	-	✓	-	-	Same Performance
	-	-	-	✓	✓			✓	-	-	✓	-	-	PerformanceDropped
	-	✓	-	-	✓	-	-	✓	-	-	✓	-	-	PerformanceDropped
2	✓	-	-	-	-	✓	-	-	✓	-	-	✓	-	PerformanceDropped
	-	✓	-	-		✓	-	-	✓	-	-	✓	-	PerformanceDropped
	-	-	✓	-		✓	-	-	✓	-	-	✓	-	PerformanceDropped
	-	-	-	✓		✓		-	✓	-	-	✓	-	PerformanceDropped
3	✓	-	-	-	-	-	✓	-	-	✓	-	-	✓	PerformanceDropped
	-	✓	-	-	-	-	✓	-	-	✓	-	-	✓	PerformanceDropped
	-	-	✓	-	-	-	✓	-	-	✓	-	-	✓	PerformanceDropped
	-	-	-	✓	-	-	✓	-	-	✓	-	-	✓	PerformanceDropped

Table 6

Performance comparison of proposed SNC_Net model with SOTA classifiers.

Ref	Year	Models	Diseases	Accuracy	Precision	Recall	F1 Score
[10]	2021	ANN	Multiclassification	95.30%	94.63%	94.87%	–
[94]	2023	VGG-13	Multiclassification	89.51%	90.68%	89.46%	90.07%
[95]	2023	Ensemble	Binary	93.00%	92.00%	94.00%	93.00%
[96]	2023	DRNN	Binary	94.29%	93.75%	95.74%	–
[97]	2022	Ensemble	Binary	95.76%	96.67%	96.99%	96.85%
[98]	2023	DSCC_Net	Binary	94.17%	94.28%	93.76%	93.93%
Ours	-	SNC_Net (with SMOTE Tomek)	Multiclassification	97.81%	98.31%	97.89%	98.10%

References

1. Naeem, A.; Anees, T. DVFNet: A deep feature fusion-based model for the multiclassification of skin cancer utilizing dermoscopy images. PLoS ONE; 2024; 20, e0297667. [DOI: https://dx.doi.org/10.1371/journal.pone.0297667]

2. American Cancer Society|Causes of Skin Cancer. Available online: https://www.cancer.org/content/dam/CRC/PDF/Public/8893.00.pdf (accessed on 4 March 2024).

3. Dilda, M.; Akra, S.; Irfa, M.; Kha, H.U.; Ramza, M.; Mahmoo, A.R.; Alsaiar, S.A.; Saee, A.H.; Alraddad, M.O.; Mahnashi, M.H. Skin cancer detection: A review using deep learning techniques. Int. J. Environ. Res. Public Health; 2021; 18, 5479. [DOI: https://dx.doi.org/10.3390/ijerph18105479]

4. Saginal, K.; Barsou, A.; Alur, J.S.; Rawla, P.; Barsouk, A. Epidemiology of melanoma. Med. Sci.; 2021; 9, 63. [DOI: https://dx.doi.org/10.3390/medsci9040063]

5. Garne, W.B.; Smit, B.D.; Ludmi, E.B.; Wakefiel, D.V.; Shabaso, J.; William, G.R.; Martin, M.Y.; Wan, Y.; Ball, M.T.; VanderWalde, N.A. Predicting future cancer incidence by age, race, ethnicity, and sex. J. Geriatr. Oncol.; 2023; 14, 101393. [DOI: https://dx.doi.org/10.1016/j.jgo.2022.10.008]

6. Eddy, K.; Chen, S. Overcoming immune evasion in melanoma. Int. J. Mol. Sci.; 2020; 21, 8984. [DOI: https://dx.doi.org/10.3390/ijms21238984]

7. Patel, R.H.; Foltz, E.A.; Witkowski, A.; Ludzik, J. Analysis of Artificial Intelligence-Based Approaches Applied to Non-Invasive Imaging for Early Detection of Melanoma: A Systematic Review. Cancers; 2023; 15, 4694. [DOI: https://dx.doi.org/10.3390/cancers15194694]

8. Cives, M.; Mannavola, F.; Lospalluti, L.; Sergi, M.C.; Cazzato, G.; Filoni, E.; Cavallo, F.; Giudice, G.; Stucci, L.S.; Porta, C. et al. Non-melanoma skin cancers: Biological and clinical features. Int. J. Mol. Sci.; 2020; 21, 5394. [DOI: https://dx.doi.org/10.3390/ijms21155394]

9. Chiang, E.; Stafford, H.; Buell, J.; Ramesh, U.; Amit, M.; Nagarajan, P.; Migden, M.; Yaniv, D. Review of the Tumor Microenvironment in Basal and Squamous Cell Carcinoma. Cancers; 2023; 15, 2453. [DOI: https://dx.doi.org/10.3390/cancers15092453]

10. Abunadi, I.; Senan, E.M. Deep Learning and machine learning techniques of diagnosis dermoscopy images for early detection of skin diseases. Electronics; 2021; 10, 3158. [DOI: https://dx.doi.org/10.3390/electronics10243158]

11. Liu, Y.; Jain, A.; Eng, C.; Way, D.H.; Lee, K.; Bui, P.; Kanada, K.; Marinho, G.d.O.; Gallegos, J.; Gabriele, S. et al. A deep learning system for differential diagnosis of skin diseases. Nat. Med.; 2020; 26, pp. 900-908. [DOI: https://dx.doi.org/10.1038/s41591-020-0842-3]

12. Da1, K.; Cockerel1, C.J.; Pati1, A.; Pietkiewic1, P.; Giulin1, M.; Grabb1, S.; Goldust, M. Machine learning and its application in skin cancer. Int. J. Environ. Res. Public Health; 2021; 18, 13409. [DOI: https://dx.doi.org/10.3390/ijerph182413409]

13. Muhaba, K.A.; Dese, K.; Aga, T.M.; Zewdu, F.T.; Simegn, G.L. Automatic skin disease diagnosis using deep learning from clinical image and patient information. Ski. Health Dis.; 2022; 2, e81. [DOI: https://dx.doi.org/10.1002/ski2.81]

14. Kim, S.-Y.; Choi, Y.; Kim, E.-K.; Han, B.-K.; Yoon, J.H.; Choi, J.S.; Chang, J.M. Deep learning-based computer-aided diagnosis in screening breast ultrasound to reduce false-positive diagnoses. Sci. Rep.; 2021; 11, 395. [DOI: https://dx.doi.org/10.1038/s41598-020-79880-0]

15. Vasconcelos, C.N.; Vasconcelos, B.N. Experiments using deep learning for dermoscopy image analysis. Pattern Recognit. Lett.; 2020; 139, pp. 95-103. [DOI: https://dx.doi.org/10.1016/j.patrec.2017.11.005]

16. Zakhem, G.A.; Fakhoury, J.W.; Motosko, C.C.; Ho, R.S. Characterizing the role of dermatologists in developing artificial intelligence for assessment of skin cancer. J. Am. Acad. Dermatol.; 2021; 85, pp. 1544-1556. [DOI: https://dx.doi.org/10.1016/j.jaad.2020.01.028]

17. Reshma, G.; Al-Atroshi, C.; Nassa, V.K.; Geetha, B.; Sunitha, G.; Galety, M.G.; Neelakandan, S. Deep learning-based skin lesion diagnosis model using dermoscopic images. Intell. Autom. Soft Comput.; 2022; 31, pp. 621-634. [DOI: https://dx.doi.org/10.32604/iasc.2022.019117]

18. Li, W.; Raj, A.N.J.; Tjahjadi, T.; Zhuang, Z. Digital hair removal by deep learning for skin lesion segmentation. Pattern Recognit.; 2021; 117, 107994. [DOI: https://dx.doi.org/10.1016/j.patcog.2021.107994]

19. Chen, R.-C.; Dewi, C.; Huang, S.-W.; Caraka, R.E. Selecting critical features for data classification based on machine learning methods. J. Big Data; 2020; 7, 52. [DOI: https://dx.doi.org/10.1186/s40537-020-00327-4]

20. Ali, A.-R.; Li, J.; Yang, G.; O’shea, S.J. A machine learning approach to automatic detection of irregularity in skin lesion border using dermoscopic images. PeerJ Comput. Sci.; 2020; 6, e268. [DOI: https://dx.doi.org/10.7717/peerj-cs.268]

21. Wang, Y.; Cai, J.; Louie, D.C.; Wang, Z.J.; Lee, T.K. Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection. Comput. Biol. Med.; 2021; 137, 104812. [DOI: https://dx.doi.org/10.1016/j.compbiomed.2021.104812]

22. Sharma, A.K.; Tiwari, S.; Aggarwal, G.; Goenka, N.; Kumar, A.; Chakrabarti, P.; Chakrabarti, T.; Gono, R.; Leonowicz, Z.; Jasinski, M. Dermatologist-level classification of skin cancer using cascaded ensembling of convolutional neural network and handcrafted features based deep neural network. IEEE Access; 2022; 10, pp. 17920-17932. [DOI: https://dx.doi.org/10.1109/ACCESS.2022.3149824]

23. Oukil, S.; Kasmi, R.; Mokrani, K.; García-Zapirain, B. Automatic segmentation and melanoma detection based on color and texture features in dermoscopic images. Ski. Res. Technol.; 2022; 28, pp. 203-211. [DOI: https://dx.doi.org/10.1111/srt.13111]

24. Saba, T. Computer vision for microscopic skin cancer diagnosis using handcrafted and non-handcrafted features. Microsc. Res. Tech.; 2021; 84, pp. 1272-1283. [DOI: https://dx.doi.org/10.1002/jemt.23686]

25. Mazhar, T.; Haq, I.; Ditta, A.; Mohsan, S.A.H.; Rehman, F.; Zafar, I.; Gansau, J.A.; Goh, L.P.W. The role of machine learning and deep learning approaches for the detection of skin cancer. Healthcare; 2023; 11, 415. [DOI: https://dx.doi.org/10.3390/healthcare11030415]

26. Sharafudeen, M.; Vinod Chandra, C.S. Detecting skin lesions fusing handcrafted features in image network ensembles. Multimed. Tools Appl.; 2023; 82, pp. 3155-3175. [DOI: https://dx.doi.org/10.1007/s11042-022-13046-0]

27. Naeem, A.; Farooq, M.S.; Khelifi, A.; Abid, A. Malignant melanoma classification using deep learning: Datasets, performance measurements, challenges and opportunities. IEEE Access; 2020; 8, pp. 110575-110597. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3001507]

28. Tump, P.P.; Kabir, M.A. An artificial neural network based detection and classification of melanoma skin cancer using hybrid texture features. Sens. Int.; 2021; 2, 100128. [DOI: https://dx.doi.org/10.1016/j.sintl.2021.100128]

29. Bassel, A.; Abdulkareem, A.B.; Alyasseri, Z.A.A.; Sani, N.S.; Mohammed, H.J. automatic malignant and benign skin cancer classification using a hybrid deep learning approach. Diagnostics; 2022; 12, 2472. [DOI: https://dx.doi.org/10.3390/diagnostics12102472]

30. Anan, V.; Gupt, S.; Kounda, D.; Naya, S.R.; Shafi, J.; Bhoi, A.K. Segmentation and Classification of Skin Cancer Using K-means Clustering and EfficientNetB0 Model. Advances in Communication, Devices and Networking: Proceedings of ICCDN 2021; Springer Nature: Singapore, 2022; pp. 471-481.

31. Toğaçar, M.; Cömert, Z.; Ergen, B. Intelligent skin cancer detection applying autoencoder, MobileNetV2 and spiking neural networks. Chaos Solitons Fractals; 2021; 144, 110714. [DOI: https://dx.doi.org/10.1016/j.chaos.2021.110714]

32. Jain, S.; Agrawal, K. An Efficient Diagnosis of Melanoma Skin Disease Using DenseNet-121. Proceedings of the 2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS); Tashkent, Uzbekistan, 1 November 2023; pp. 908-912.

33. Demir, A.; Yilmaz, F.; Kose, O. Early detection of skin cancer using deep learning architectures: Resnet-101 and inception-v3. Proceedings of the 2019 Medical Technologies Congress (TIPTEKNO); Izmir, Turkey, 3–5 October 2019; pp. 1-4.

34. Naeem, A.; Anees, T.; Ahmed, K.T.; Naqvi, R.A.; Ahmad, S.; Whangbo, T. Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval. Complex Intell. Syst.; 2022; 9, pp. 1729-1751. [DOI: https://dx.doi.org/10.1007/s40747-022-00866-8]

35. Bakheet, S. An SVM Framework for Malignant Melanoma Detection Based on Optimized HOG Features. Computation; 2017; 5, 4. [DOI: https://dx.doi.org/10.3390/computation5010004]

36. Khan, M.Q.; Hussain, A.; Rehman, S.U.; Khan, U.; Maqsood, M.; Mehmood, K.; Khan, M.A. Classification of melanoma and nevus in digital images for diagnosis of skin cancer. IEEE Access; 2019; 7, pp. 90132-90144. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2926837]

37. Warsi, F.; Khanam, R.; Kamya, S.; Suárez-Araujo, C.P. An efficient 3D color-texture feature and neural network technique for melanoma detection. Inform. Med. Unlocked; 2019; 17, 100176. [DOI: https://dx.doi.org/10.1016/j.imu.2019.100176]

38. Kumar, M.; Alshehri, M.; AlGhamdi, R.; Sharma, P.; Deep, V. A DE-ANN Inspired skin cancer detection approach using fuzzy c-means clustering. Mob. Netw. Appl.; 2020; 25, pp. 1319-1329. [DOI: https://dx.doi.org/10.1007/s11036-020-01550-2]

39. Akan, H.; Yıldız, M.Z. Development of new descriptor for melanoma detection on dermoscopic images. Med. Biol. Eng. Comput.; 2020; 58, pp. 2711-2723. [DOI: https://dx.doi.org/10.1007/s11517-020-02248-z]

40. Codella, N.C.F.; Nguyen, Q.-B.; Pankanti, S.; Gutman, D.A.; Helba, B.; Halpern, A.C.; Smith, J.R. Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J. Res. Dev.; 2017; 61, pp. 5:1-5:15. [DOI: https://dx.doi.org/10.1147/JRD.2017.2708299]

41. L1, X.; W1, J.; Jian1, H.; Che1, E.Z.; Dong, X.; Rong, R. Skin lesion classification via combining deep learning features and clinical criteria representations. BioRxiv; 2018; 382010.

42. Khan, M.A.; Sharif, M.I.; Raza, M.; Anjum, A.; Saba, T.; Shad, S.A. Skin lesion segmentation and classification: A unified framework of deep neural network features fusion and selection. Expert Syst.; 2022; 39, 12497. [DOI: https://dx.doi.org/10.1111/exsy.12497]

43. Almaraz-Damian, J.-A.; Ponomaryov, V.; Sadovnychiy, S.; Castillejos-Fernandez, H. Melanoma and nevus skin lesion classification using handcraft and deep learning feature fusion via mutual information measures. Entropy; 2020; 22, 484. [DOI: https://dx.doi.org/10.3390/e22040484]

44. Jayapriya, K.; Jacob, I.J. Hybrid fully convolutional networks-based skin lesion segmentation and melanoma detection using deep feature. Int. J. Imaging Syst. Technol.; 2020; 30, pp. 348-357. [DOI: https://dx.doi.org/10.1002/ima.22377]

45. Kumar Tiwar, A.; Kumar Mishr, M.; Ranjan Panda, A.; Panda, B. Hosmi-lbp-Based Feature Extraction for Melanoma Detection Using Hybrid Deep Learning Models. J. Mech. Med. Biol.; 2021; 21, 2150029. [DOI: https://dx.doi.org/10.1142/S0219519421500299]

46. Salid1, J.A.; Ruiz, C. Using deep learning to detect melanoma in dermoscopy images. Int. J. Mach. Learn. Comput.; 2018; 8, pp. 61-68. [DOI: https://dx.doi.org/10.18178/ijmlc.2018.8.1.664]

47. Dorj, U.-O.; Lee, K.-K.; Choi, J.-Y.; Lee, M. The skin cancer classification using deep convolutional neural network. Multimed. Tools Appl.; 2018; 77, pp. 9909-9924. [DOI: https://dx.doi.org/10.1007/s11042-018-5714-1]

48. Shahi, A.H.; Kamal, A.; Elattar, M.A. Deep ensemble learning for skin lesion classification from dermoscopic images. Proceedings of the 2018 9th Cairo International Biomedical Engineering Conference (CIBEC); Cairo, Egypt, 20–22 December 2018; pp. 150-153.

49. Dascalu, A.; David, E. Skin cancer detection by deep learning and sound analysis algorithms: A prospective clinical study of an elementary dermoscope. EBioMedicine; 2019; 43, pp. 107-113. [DOI: https://dx.doi.org/10.1016/j.ebiom.2019.04.055]

50. Pham, H.N.; Koay, C.Y.; Chakraborty, T.; Gupta, S.; Tan, B.L.; Wu, H.; Vardhan, A.; Nguyen, Q.H.; Palaparthi, N.R.; Nguyen, B.P. et al. Lesion Segmentation and Automated Melanoma Detection using Deep Convolutional Neural Networks and XGBoost. Proceedings of the 2019 International Conference on System Science and Engineering (ICSSE); Dong Hoi, Vietnam, 20–21 July 2019; pp. 142-147.

51. Bisl1, D.; Choromansk1, A.; Berma1, R.S.; Stei1, J.A.; Polsky, D. Towards automated melanoma detection with deep learning: Data purification and augmentation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops 2019; Long Beach, CA, USA, 15–20 June 2019.

52. Brinker, T.J.; Hekler, A.; Enk, A.H.; Berking, C.; Haferkamp, S.; Hauschild, A.; Weichenthal, M.; Klode, J.; Schadendorf, D.; Holland-Letz, T. et al. Deep neural networks are superior to dermatologists in melanoma image classification. Eur. J. Cancer; 2019; 119, pp. 11-17. [DOI: https://dx.doi.org/10.1016/j.ejca.2019.05.023]

53. Carcagnì, P.; Leo, M.; Cuna, A.; Mazzeo, P.L.; Spagnolo, P.; Celeste, G.; Distante, C. Classification of Skin Lesions by Combining Multilevel Learnings in a DenseNet Architecture. Proceedings of the 20th International Conference Image Analysis and Processing (ICIAP 2019); Trento, Italy, 9–13 September 2019; pp. 335-344. [DOI: https://dx.doi.org/10.1007/978-3-030-30642-7_30]

54. Sarkar, R.; Chatterjee, C.C.; Hazra, A. Diagnosis of melanoma from dermoscopic images using a deep depthwise separable residual convolutional network. IET Image Process.; 2019; 13, pp. 2130-2142. [DOI: https://dx.doi.org/10.1049/iet-ipr.2018.6669]

55. El-Khatib, H.; Popescu, D.; Ichim, L. Deep learning–based methods for automatic diagnosis of skin lesions. Sensors; 2020; 20, 1753. [DOI: https://dx.doi.org/10.3390/s20061753]

56. Adegun, A.A.; Viriri, S. FCN-based DenseNet Framework for automated detection and classification of skin lesions in dermoscopy images. IEEE Access; 2020; 8, pp. 150377-150396. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3016651]

57. Kumar, N.M.S.; Hariprasath, K.; Tamilselvi, S.; Kavinya, A.; Kaviyavarshini, N. Detection of stages of melanoma using deep learning. Multimedia Tools Appl.; 2021; 80, pp. 18677-18692. [DOI: https://dx.doi.org/10.1007/s11042-021-10572-1]

58. Al1, M.S.; Mia1, M.S.; Haqu1, J.; Rahma1, M.M.; Islam, M.K. An enhanced technique of skin cancer classification using deep convolutional neural network with transfer learning models. Mach. Learn. Appl.; 2021; 5, 100036.

59. Sevli, O. A deep convolutional neural network-based pigmented skin lesion classification application and experts evaluation. Neural Comput. Appl.; 2021; 33, pp. 12039-12050. [DOI: https://dx.doi.org/10.1007/s00521-021-05929-4]

60. Jojoa Acost, M.F.; Caballero Tova, L.Y.; Garcia-Zapirai, M.B.; Percybrooks, W.S. Melanoma diagnosis using deep learning techniques on dermatoscopic images. BMC Med. Imaging; 2021; 21, 6. [DOI: https://dx.doi.org/10.1186/s12880-020-00534-8]

61. Brinker, T.J.; Hekler, A.; Enk, A.H.; Klode, J.; Hauschild, A.; Berking, C.; Schilling, B.; Haferkamp, S.; Schadendorf, D.; Holland-Letz, T. et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur. J. Cancer; 2019; 113, pp. 47-54. [DOI: https://dx.doi.org/10.1016/j.ejca.2019.04.001]

62. Khan, M.A.; Akram, T.; Zhang, Y.-D.; Sharif, M. Attributes based skin lesion detection and recognition: A mask RCNN and transfer learning-based deep learning framework. Pattern Recognit. Lett.; 2021; 143, pp. 58-66. [DOI: https://dx.doi.org/10.1016/j.patrec.2020.12.015]

63. Khan, M.A.; Zhang, Y.-D.; Sharif, M.; Akram, T. Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification. Comput. Electr. Eng.; 2021; 90, 106956. [DOI: https://dx.doi.org/10.1016/j.compeleceng.2020.106956]

64. He, X.; Wang, Y.; Zhao, S.; Chen, X. Co-Attention Fusion Network for Multimodal Skin Cancer Diagnosis. Pattern Recognit.; 2023; 133, 108990. [DOI: https://dx.doi.org/10.1016/j.patcog.2022.108990]

65. Nawaz, M.; Mehmood, Z.; Nazir, T.; Naqvi, R.A.; Rehman, A.; Iqbal, M.; Saba, T. Skin cancer detection from dermoscopic images using deep learning and fuzzy k-means clustering. Microsc. Res. Tech.; 2022; 85, pp. 339-351. [DOI: https://dx.doi.org/10.1002/jemt.23908]

66. Mohakud, R.; Dash, R. Skin cancer image segmentation utilizing a novel EN-GWO based hyper-parameter optimized FCEDN. J. King Saud Univ.-Comput. Inf. Sci.; 2022; 34, pp. 9889-9904. [DOI: https://dx.doi.org/10.1016/j.jksuci.2021.12.018]

67. Mukadam, S.B.; Patil, H.Y. Skin Cancer Classification Framework Using Enhanced Super Resolution Generative Adversarial Network and Custom Convolutional Neural Network. Appl. Sci.; 2023; 13, 1210. [DOI: https://dx.doi.org/10.3390/app13021210]

68. Afza, F.; Sharif, M.; Mittal, M.; Khan, M.A.; Jude Hemanth, D. A hierarchical three-step superpixels and deep learning framework for skin lesion classification. Methods; 2022; 202, pp. 88-102. [DOI: https://dx.doi.org/10.1016/j.ymeth.2021.02.013]

69. Sayed, G.I.; Soliman, M.M.; Hassanien, A.E. A novel melanoma prediction model for imbalanced data using optimized SqueezeNet by bald eagle search optimization. Comput. Biol. Med.; 2021; 136, 104712. [DOI: https://dx.doi.org/10.1016/j.compbiomed.2021.104712]

70. Alenezi, F.; Armghan, A.; Polat, K. A multi-stage melanoma recognition framework with deep residual neural network and hyperparameter optimization-based decision support in dermoscopy images. Expert Syst. Appl.; 2023; 215, 119352. [DOI: https://dx.doi.org/10.1016/j.eswa.2022.119352]

71. Jasil, S.P.G.; Ulagamuthalvi, V. A hybrid CNN architecture for skin lesion classification using deep learning. Soft Comput.; 2023; pp. 1-10. [DOI: https://dx.doi.org/10.1007/s00500-023-08035-w]

72. Bindhu, A.; Thanammal, K.K. Multi-stage feature extraction-based classification of skin cancer detection. Soft Comput.; 2023; pp. 1-14. [DOI: https://dx.doi.org/10.1007/s00500-023-08611-0]

73. Tabrizchi, H.; Parvizpour, S.; Razmara, J. An Improved VGG Model for Skin Cancer Detection. Neural Process. Lett.; 2022; 55, pp. 3715-3732. [DOI: https://dx.doi.org/10.1007/s11063-022-10927-1]

74. Ala, M.J.; Mohamma, M.S.; Hossai, M.A.; Showmi, I.A.; Raiha, M.S.; Ahme, S.; Mahmud, T.I. S2C-DeLeNet: A parameter transfer based segmentation-classification integration for detecting skin cancer lesions from dermoscopic images. Comput. Biol. Med.; 2022; 150, 106148.

75. Shett, B.; Fernande, R.; Rodrigue, A.P.; Chengode, R.; Bhattachary, S.; Lakshmanna, K. Skin lesion classification of dermoscopic images using machine learning and convolutional neural network. Sci. Rep.; 2022; 12, 18134. [DOI: https://dx.doi.org/10.1038/s41598-022-22644-9]

76. Dong, C.; Dai, D.; Zhang, Y.; Zhang, C.; Li, Z.; Xu, S. Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification. Comput. Biol. Med.; 2023; 152, 106321. [DOI: https://dx.doi.org/10.1016/j.compbiomed.2022.106321]

77. Qureshi, A.S.; Roos, T. Transfer learning with ensembles of deep neural networks for skin cancer detection in imbalanced data sets. Neural Process. Lett.; 2023; 55, pp. 4461-4479. [DOI: https://dx.doi.org/10.1007/s11063-022-11049-4]

78. Panthakka, A.; Anza, S.M.; Jama, S.; Mansoor, W. Concatenated Xception-ResNet50—A novel hybrid approach for accurate skin cancer prediction. Comput. Biol. Med.; 2022; 150, 106170.

79. Akilandasowmya, G.; Nirmaladevi, G.; Suganthi, S.; Aishwariya, A. Skin cancer diagnosis: Leveraging deep hidden features and ensemble classifiers for early detection and classification. Biomed. Signal Process. Control.; 2024; 88, 105306. [DOI: https://dx.doi.org/10.1016/j.bspc.2023.105306]

80. Cassidy, B.; Kendrick, C.; Brodzicki, A.; Jaworek-Korjakowska, J.; Yap, M.H. Analysis of the ISIC image datasets: Usage, benchmarks and recommendations. Med. Image Anal.; 2022; 75, 102305. [DOI: https://dx.doi.org/10.1016/j.media.2021.102305]

81. Malik, H.; Naeem, A.; Naqvi, R.A.; Loh, W.-K. DMFL_Net: A Federated Learning-Based Framework for the Classification of COVID-19 from Multiple Chest Diseases Using X-rays. Sensors; 2023; 23, 743. [DOI: https://dx.doi.org/10.3390/s23020743]

82. Khan, A.H.; Iskandar, D.A.; Al-Asad, J.F.; El-Nakla, S. Classification of skin lesion with hair and artifacts removal using Black-hat morphology and total variation. Int. J. Comput. Digit. Syst.; 2021; 10, pp. 597-604. [DOI: https://dx.doi.org/10.12785/ijcds/100157]

83. Bansal, P.; Garg, R.; Soni, P. Detection of melanoma in dermoscopic images by integrating features extracted using handcrafted and deep learning models. Comput. Ind. Eng.; 2022; 168, 108060. [DOI: https://dx.doi.org/10.1016/j.cie.2022.108060]

84. Sekhar, K.S.R.; Babu, T.R.; Prathibha, G.; Vijay, K.; Ming, L.C. Dermoscopic image classification using CNN with Handcrafted features. J. King Saud Univ.-Sci.; 2021; 33, 101550. [DOI: https://dx.doi.org/10.1016/j.jksus.2021.101550]

85. Maqsood, S.; Damaševičius, R. Multiclass skin lesion localization and classification using deep learning based features fusion and selection framework for smart healthcare. Neural Netw.; 2023; 160, pp. 238-258. [DOI: https://dx.doi.org/10.1016/j.neunet.2023.01.022]

86. Thepad, S.D.; Abin, D.; Dhake, A.R. Machine Learning Based Melanoma Skin Cancer Detection using Fusion of Thepade’s SBTC and GLCM Features. Proceedings of the 2021 International Conference on Disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON); Bengaluru, India, 19–21 November 2021; IEEE: Piscataway, NJ, USA, 2021; Volume 1, pp. 300-303.

87. Grignaffini, F.; Barbuto, F.; Piazzo, L.; Troiano, M.; Simeoni, P.; Mangini, F.; Pellacani, G.; Cantisani, C.; Frezza, F. Machine Learning Approaches for Skin Cancer Classification from Dermoscopic Images: A Systematic Review. Algorithms; 2022; 15, 438. [DOI: https://dx.doi.org/10.3390/a15110438]

88. Singh, S.; Alam, M.; Singh, B. Orthogonal moment feature extraction and classification of melanoma images. J. Inf. Optim. Sci.; 2020; 41, pp. 195-203. [DOI: https://dx.doi.org/10.1080/02522667.2020.1721585]

89. Chaturvedi, S.S.; Tembhurne, J.V.; Diwan, T. A multi-class skin Cancer classification using deep convolutional neural networks. Multimed. Tools Appl.; 2020; 79, pp. 28477-28498. [DOI: https://dx.doi.org/10.1007/s11042-020-09388-2]

90. Nahata, H.; Singh, S.P. Deep learning solutions for skin cancer detection and diagnosis. Machine Learning with Health Care Perspective: Machine Learning and Healthcare; Springer: Cham, Switzerland, 2020; pp. 159-182.

91. Thurnhofer-Hemsi, K.; Domínguez, E. A Convolutional neural network framework for accurate skin cancer detection. Neural Process. Lett.; 2021; 53, pp. 3073-3093. [DOI: https://dx.doi.org/10.1007/s11063-020-10364-y]

92. Sha, A.; Sha, M.; Pandy, A.; Sushr, R.; Sushr, R.; Meht, M.; Pate, K.; Patel, K. A comprehensive study on skin cancer detection using artificial neural network (ANN) and convolutional neural network (CNN). Clin. eHealth; 2023; 6, pp. 76-84. [DOI: https://dx.doi.org/10.1016/j.ceh.2023.08.002]

93. Höhn, J.; Krieghoff-Henning, E.; Jutzi, T.B.; von Kalle, C.; Utikal, J.S.; Meier, F.; Gellrich, F.F.; Hobelsberger, S.; Hauschild, A.; Schlager, J.G. et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur. J. Cancer; 2021; 149, pp. 94-101. [DOI: https://dx.doi.org/10.1016/j.ejca.2021.02.032]

94. Qasim Gilan, S.; Sye, T.; Umai, M.; Marques, O. Skin Cancer Classification Using Deep Spiking Neural Network. J. Digit. Imaging; 2023; 36, pp. 1137-1147. [DOI: https://dx.doi.org/10.1007/s10278-023-00776-2]

95. Tembhurne, J.V.; Hebbar, N.; Patil, H.Y.; Diwan, T. Skin cancer detection using ensemble of machine learning and deep learning techniques. Multimed. Tools Appl.; 2023; 82, pp. 27501-27524. [DOI: https://dx.doi.org/10.1007/s11042-023-14697-3]

96. Alenezi, F.; Armghan, A.; Polat, K. Wavelet transform based deep residual neural network and ReLU based Extreme Learning Machine for skin lesion classification. Expert Syst. Appl.; 2023; 213, 119064. [DOI: https://dx.doi.org/10.1016/j.eswa.2022.119064]

97. Shorfuzzaman, M. An explainable stacked ensemble of deep learning models for improved melanoma skin cancer detection. Multimed. Syst.; 2022; 28, pp. 1309-1323. [DOI: https://dx.doi.org/10.1007/s00530-021-00787-5]

98. Tahir, M.; Naeem, A.; Malik, H.; Tanveer, J.; Naqvi, R.A.; Lee, S.-W. DSCC_Net: Multi-Classification Deep Learning Models for Diagnosing of Skin Cancer Using Dermoscopic Images. Cancers; 2023; 15, 2179. [DOI: https://dx.doi.org/10.3390/cancers15072179]

99. Riaz, S.; Naeem, A.; Malik, H.; Naqvi, R.A.; Loh, W.-K. Federated and Transfer Learning Methods for the Classification of Melanoma and Nonmelanoma Skin Cancers: A Prospective Study. Sensors; 2023; 23, 8457. [DOI: https://dx.doi.org/10.3390/s23208457] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37896548]

100. Naeem, A.; Anees, T.; Fiza, M.; Naqvi, R.A.; Lee, S.-W. SCDNet: A Deep Learning-Based Framework for the Multiclassification of Skin Cancer Using Dermoscopy Images. Sensors; 2022; 22, 5652. [DOI: https://dx.doi.org/10.3390/s22155652] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35957209]