Abstract: Skin cancer is one of the most common malignant tumors.Early diagnosis and active treatment can significantly improve the survival rate of patients.The existing research on automatic diagnosis of skin diseases based on convolutional neural networks is devoted to developing a deeper architecture,andthe increase in computational resource overhead and model parameter count subsequently restricts the lightweight deployment of the model.At the same time,there is an imbalance in the distribution of skin disease data,which leads to a decrease in the performance of the model in identifying rare diseases.Aiming at the dual challenges of high computational complexity and unbalanced data distribution faced by the current skin disease automatic diagnosis system,this paper proposes a hierarchical collaborative knowledge distillation method HCKD.By jointly optimizing the response knowledge distillation of the fully connected layer,the structural relationship knowledge distillation of the embedded la-yer,and the channel feature knowledge distillation of the convolutional layer,a hierarchical knowledge transfer mechanism is constructed to achieve efficient compression of the model.At the same time,a weighted cross-entropy loss function is introduced to enhance the recognition ability of the model for rare diseases.In the classification task of the ISIC 2019 dataset,the student model trained by the HCKD method achieves an accuracy rate of 85.7% and a balance accuracy rate of 82.6%,and the number of model parameters and computational resource overhead are significantly lower than the teacher model.The ability to identify rare diseases has been improved,and the best results have been achieved compared with the current three popular knowledge distillation methods.

Key words: Dermoscopic images, Deep learning, Convolutional neural network, Knowledge distillation, Lightweight deployment