HCKD:基于皮肤镜图像的轻量级皮肤病变分类方法

doi:10.11896/jsjkx.250600143

摘要/Abstract

摘要： 皮肤癌是最常见的恶性肿瘤之一,早期诊断并积极治疗能显著提高患者的生存率。现有的基于卷积神经网络的皮肤病自动诊断研究致力于开发更深层的架构,计算资源开销和模型参数量随之增加,限制了模型的轻量化部署。同时皮肤病数据存在分布不平衡问题,导致模型在识别罕见病类时性能下降。针对当前皮肤病自动诊断系统面临的计算复杂度高和数据分布不平衡的双重挑战,提出了层次协同知识蒸馏方法 HCKD(Hierarchical Collaborative Knowledge Distillation),通过联合优化全连接层的响应知识蒸馏、嵌入层的结构关系知识蒸馏和卷积层的通道特征知识蒸馏,构建层次化知识迁移机制,实现模型高效压缩,同时引入加权交叉熵损失函数以增强模型对罕见病类的识别能力。在ISIC 2019数据集的分类任务中,HCKD方法训练的学生模型取得了85.7%的准确率、82.6%的平衡准确率,且模型参数量和计算资源开销较教师模型大幅降低,识别罕见病类能力得到提升,与当前3种流行的知识蒸馏方法相比达到了最佳效果。

关键词: 皮肤镜图像, 深度学习, 卷积神经网络, 知识蒸馏, 轻量级部署

Abstract: Skin cancer is one of the most common malignant tumors.Early diagnosis and active treatment can significantly improve the survival rate of patients.The existing research on automatic diagnosis of skin diseases based on convolutional neural networks is devoted to developing a deeper architecture,andthe increase in computational resource overhead and model parameter count subsequently restricts the lightweight deployment of the model.At the same time,there is an imbalance in the distribution of skin disease data,which leads to a decrease in the performance of the model in identifying rare diseases.Aiming at the dual challenges of high computational complexity and unbalanced data distribution faced by the current skin disease automatic diagnosis system,this paper proposes a hierarchical collaborative knowledge distillation method HCKD.By jointly optimizing the response knowledge distillation of the fully connected layer,the structural relationship knowledge distillation of the embedded la-yer,and the channel feature knowledge distillation of the convolutional layer,a hierarchical knowledge transfer mechanism is constructed to achieve efficient compression of the model.At the same time,a weighted cross-entropy loss function is introduced to enhance the recognition ability of the model for rare diseases.In the classification task of the ISIC 2019 dataset,the student model trained by the HCKD method achieves an accuracy rate of 85.7% and a balance accuracy rate of 82.6%,and the number of model parameters and computational resource overhead are significantly lower than the teacher model.The ability to identify rare diseases has been improved,and the best results have been achieved compared with the current three popular knowledge distillation methods.

Key words: Dermoscopic images, Deep learning, Convolutional neural network, Knowledge distillation, Lightweight deployment

中图分类号:

TP391

李思雨, 钱文华. HCKD:基于皮肤镜图像的轻量级皮肤病变分类方法[J]. 计算机科学, 2026, 53(6A): 250600143-9. https://doi.org/10.11896/jsjkx.250600143

LI Siyu, QIAN Wenhua. HCKD:Lightweight Skin Lesion Classification Method Based on Dermoscopic Images[J]. Computer Science, 2026, 53(6A): 250600143-9. https://doi.org/10.11896/jsjkx.250600143

参考文献

[1] ARNOLD M,SINGH D,LAVERSANNE M,et al.Global burden of cutaneous melanoma in 2020 and projections to 2040[J].JAMA Dermatology,2022,158(5):495-503.
[2] PATEL R H,FOLTZ E A,WITKOWSKI A,et al.Analysis of artificial intelligence-based approaches applied to non-invasive imaging for early detection of melanoma:a systematic review[J].Cancers,2023,15(19):4694.
[3] OLAYAH F,SENAN E M,AHMED I A,et al.AI techniques ofdermoscopy image analysis for the early detection of skin lesions based on combined CNN features[J].Diagnostics,2023,13(7):1314.
[4] GAJERA H K,NAYAK D R,ZAVERI M A.A comprehensive analysis of dermoscopy images for melanoma detection via deep CNN features[J].Biomedical Signal Processing and Control,2023,79:104186.
[5] NACHBAR F,STOLZ W,MERKLE T,et al.The ABCD rule of dermatoscopy:high prospective value in the diagnosis of doubtful melanocytic skin lesions[J].Journal of the American Academy of Dermatology,1994,30(4):551-559.
[6] WANG J,WANG S,ZHANG Y.Deep learning on medicalimage analysis[J].CAAI Transactions on Intelligence Technology,2025,10(1):1-35.
[7] MALL P K,SINGH P K,SRIVASTAV S,et al.A comprehensive review of deep neural networks for medical image processing:Recent developments and future opportunities[J].Healthcare Analytics,2023,4:100216.
[8] SALAHUDDIN Z,WOODRUFF H C,CHATTERJEE A,et al.Transparency of deep neural networks for medical image analysis:A review of interpretability methods[J].Computers in Bio-logy and Medicine,2022,140:105111.
[9] TSUNEKI M.Deep learning models in medical image analysis[J].Journal of Oral Biosciences,2022,64(3):312-320.
[10] WANG Y,CAI J,LOUIE D C,et al.Incorporating clinicalknowledge with constrained classifier chain into a multimodal deep network for melanoma detection[J].Computers in Biology and Medicine,2021,137:104812.
[11] KAUR A.Revolutionizing Dermatology with High-Accuracy Skin Condition Classification Through ResNet50[C]//2024 5th IEEE Global Conference for Advancement in Technology(GCAT).IEEE,2024:1-5.
[12] HINTON G,VINYALS O,DEAN J.Distilling the knowledge in a neural network[J].arXiv:1503.02531,2015.
[13] KHAN M S,ALAM K N,DHRUBA A R,et al.Knowledge dis-tillation approach towards melanoma detection[J].Computers in Biology and Medicine,2022,146:105581.
[14] YIM J,JOO D,BAE J,et al.A gift from knowledge distillation:Fast optimization,network minimization and transfer learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4133-4141.
[15] WANG L,YOON K J.Knowledge distillation and student-teacher learning for visual intelligence:A review and new outlooks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(6):3048-3068.
[16] GOU J,YU B,MAYBANK S J,et al.Knowledge distillation:A survey[J].International Journal of Computer Vision,2021,129(6):1789-1819.
[17] QIN D,BU J J,LIU Z,et al.Efficient medical image segmentation based on knowledge distillation[J].IEEE Transactions on Medical Imaging,2021,40(12):3820-3831.
[18] CHEN W,GAO L,LI X,et al.Lightweight convolutional neural network with knowledge distillation for cervical cells classification[J].Biomedical Signal Processing and Control,2022,71:103177.
[19] KHAN M S,ALAM K N,DHRUBA A R,et al.Knowledge distillation approach towards melanoma detection[J].Computers in Biology and Medicine,2022,146:105581.
[20] ESTEVA A,KUPREL B,NOVOA R A,et al.Dermatologist-level classification of skin cancer with deep neural networks[J].Nature,2017,542(7639):115-118.
[21] KAWAHARA J,DANESHVAR S,ARGENZIANO G,et al.Seven-point checklist and skin lesion classification using multitask multimodal neural nets[J].IEEE Journal of Biomedical and Health Informatics,2018,23(2):538-546.
[22] OU C,ZHOU S,YANG R,et al.A deep learning based multimodal fusion model for skin lesion diagnosis using smartphone collected clinical images and metadata[J].Frontiers in Surgery,2022,9:1029991.
[23] MAHAJAN K,SHARMA M,VIG L.Meta-dermdiagnosis:Few-shot skin disease identification using meta-learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2020:730-731.
[24] YAO P,SHEN S,XU M,et al.Single model deep learning on imbalanced small datasets for skin lesion classification[J].IEEE Transactions on Medical Mmaging,2021,41(5):1242-1254.
[25] WENG F,MA Y,SUN J,et al.An interpretable imbalancedsemi-supervised deep learning framework for improving differential diagnosis of skin diseases[J].arXiv:2211.10858,2022.
[26] ROMERO A,BALLAS N,KAHOU S E,et al.Fitnets:Hintsfor thin deep nets[J].arXiv:1412.6550,2014.
[27] PARK W,KIM D,LU Y,et al.Relational knowledge distillation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3967-3976.
[28] LIU Y,CAO J,LI B,et al.Knowledge distillation via instance relationship graph[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2019:7096-7104.
[29] SU C P,TSENG C H,LEE S J.Knowledge From the Dark Side:Entropy-Reweighted Knowledge Distillation for Balanced Knowledge Transfer[J].arXiv:2311.13621,2023.
[30] BACK S,LEE S,SHIN S,et al.Robust skin disease classification by distilling deep neural network ensemble for the mobile diagnosis of herpes zoster[J].IEEE Access,2021,9:20156-20169.
[31] QIN D,BU J J,LIU Z,et al.Efficient medical image segmentation based on knowledge distillation[J].IEEE Transactions on Medical Imaging,2021,40(12):3820-3831.
[32] GONG X,LI S,BAO Y,et al.Federated learning via input-output collaborative distillation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:22058-22066.
[33] KHAN M S,ALAM K N,DHRUBA A R,et al.Knowledge distillation approach towards melanoma detection[J].Computers in Biology and Medicine,2022,146:105581.
[34] GE H,HAO L,XU Z,et al.ClinKD:Cross-Modal Clinic Knowledge Distiller For Multi-Task Medical Images[J].arXiv:2502.05928,2025.
[35] CHO S,PARK S,LIM C.Area-wise relational knowledge distillation[J].Communications for Statistical Applications and Methods,2023,30(5):501-516.
[36] HUBER P J.Robust estimation of a location parameter[M]//Breakthroughs in statistics:Methodology and distribution.New York,NY:Springer New York,1992:492-518.
[37] LIU L,HUANG Q,LIN S,et al.Exploring inter-channel correlation for diversity-preserved knowledge distillation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:8271-8280.
[38] LIN M,CHEN Q,YAN S.Network in network[J].arXiv:1312.4400,2013.
[39] WANG Y,WANG Y,CAI J,et al.Ssd-kd:A self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic images[J].Medical Image Analysis,2023,84:102693.
[40] DING L,WANG Y,YUAN K,et al.Towards universal physical attacks on single object tracking[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:1236-1245.
[41] NIGAR N,UMAR M,SHAHZAD M K,et al.A deep learning approach based on explainable artificial intelligence for skin lesion classification[J].IEEE Access,2022,10:113715-113725.
[42] AYAS S.Multiclass skin lesion classification in dermoscopic images using swin transformer model[J].Neural Computing and Applications,2023,35(9):6713-6722.
[43] ZENG X,JI Z,ZHANG H,et al.DSP-KD:dual-stage progressive knowledge distillation for skin disease classification[J].Bioengineering,2024,11(1):70.
[44] ANGGRIANDI D,SUNYOTO A.Multi-class classification ofskin diseases using ResNet50[C]//2023 6th International Conference on Information and Communications Technology(ICOIACT).IEEE,2023:349-354.
[45] KAUR J,RANI S,SINGH G.A Dual-Power Approach for Skin Disease Classification Using ResNet-50 and SVM[C]//2024 Global Conference on Communications and Information Technologies(GCCIT).IEEE,2024:1-6.
[46] YUAN L,TAY F E H,LI G,et al.Revisiting knowledge distillation via label smoothing regularization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3903-3911.
[47] QIN Y,TANG Q,XIN J,et al.A Rapid identification technique of moving loads based on MobileNetV2 and transfer learning[J].Buildings,2023,13(2):572.
[48] WANG H,QI Q,SUN W,et al.Classification of skin lesionswith generative adversarial networks and improved MobileNetV2[J].International Journal of Imaging Systems and Technology,2023,33(5):1561-1576.
[49] WANG D,WANG X,WANG L,et al.A real-world dataset and benchmark for foundation model adaptation in medical image classification[J].Scientific Data,2023,10(1):574.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed