Cross-Modality Robustness of Explainable Deep Learning for Melanoma Detection: Can Dermoscopy-Trained Convolutional Neural Networks Generalise to Smartphone Images?

Introduction. Melanoma is the most lethal common skin cancer, and convolutional neural networks (CNNs) now match dermatologist accuracy on dermoscopic images. However, patients without access to a dermatologist or dermoscope photograph their lesions with smartphones, and it remains unclear whether dermoscopy-trained models — and the saliency explanations that justify their decisions — stay reliable under this change of imaging modality.

Methods. We benchmark dermoscopy-trained CNNs (ResNet-50, EfficientNet-B3, DenseNet-121) on real smartphone and paired clinical images (PAD-UFES-20, Derm7pt) and on dermoscopic images systematically degraded toward smartphone capture conditions (focus blur, illumination and colour shift, JPEG compression, lesion scale) using a custom Modality-Shift Simulator. Alongside diagnostic accuracy (melanoma sensitivity, AUROC) and calibration (ECE), we quantify explanation faithfulness (deletion and insertion) and lesion localisation (IoU, pointing game), analysed separately for melanoma and benign lesions, and combine the four axes into a composite Cross-Modality Reliability Score (CMRS). Paired differences are assessed with DeLong and McNemar tests and bootstrap confidence intervals.

Results. Preliminary experiments indicate a substantial decline in melanoma sensitivity and AUROC when models trained on dermoscopy are applied to smartphone-like images, while explanation faithfulness and localisation degrade at least as strongly as accuracy; explanation drift is associated with misclassification, most markedly for the melanoma class. Full quantitative results across all architectures and conditions will be presented at the conference.

Conclusions. Under modality shift, confident predictions and visually plausible explanations can both mislead. The CMRS provides a measurable reliability signal for assessing dermoscopy-trained models before smartphone-based triage, and the accompanying open-source evaluation tool supports reproducible assessment on new models and datasets.