Transformer-Based Ensemble Learning for Robust Multi-Class Melanoma Classification

Abstract

Melanoma signifies the most hazardous form of skin cancer, characterized by a worldwide increase in incidence despite the improvement of dermatological screening techniques. The existing techniques of automated detection have been found to be limited in terms of global context modeling, reliance on a single architecture, and the lack of clinically interpretable models. The present research aims to bridge the existing gap by introducing a novel framework of transformer-based ensemble learning for the automated detection of melanoma, combining Vision Transformer models and CNN models via the application of a meta-learning approach. In the present study, the Vision Transformer, Swin Transformer, MaxViT, and ConvNeXt models have been combined with a ResNet50 CNN model via a Random Forest-based meta-classification approach. A balanced multi-source dataset was created by combining the PH2, MedNode, and Derm7pt datasets, covering six different classes of lesions. The images were integrated using a real image-based oversampling technique. The stacked ensemble models achieved a macro F1 score of 0.998, Balanced Accuracy of 0.999, and a macro-AUC of 0.9999, indicating a rise of 12% over the highest performing model. The statistical significance of the models' complementary errors was assessed using McNemar’s test, which indicated a statistically significant complementary error for all models (p < 0.001). The Grad-CAM and GradCAM++ approaches have been modified for the different models to ensure the provision of spatially interpretable visualizations in accordance with dermatological diagnostic criteria. The proposed framework has shown that architecturally diverse stacking ensembles, when used with appropriate balancing of the dataset and leakage-resistant cross-validation, can achieve robust multi-class classification of skin lesion images with potential to inform clinical decision-making.