| Autor: |
Zeng X; Hebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China., Ji Z; Hebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China.; Telecommunications Research Centre (TRC), University of Limerick, V94 T9PX Limerick, Ireland., Zhang H; Department of Computing, Xi'an Jiaotong-Liverpool University, Suzhou 215000, China., Chen R; Changgeng Hospital, Institute for Precision Medicine, Tsinghua University, Beijing 100084, China., Liao Q; Changgeng Hospital, Institute for Precision Medicine, Tsinghua University, Beijing 100084, China., Wang J; Beijing National Research Center for Information Science and Technology, Institute for Precision Medicine, Tsinghua University, Beijing 100084, China., Lyu T; Changgeng Hospital, Institute for Precision Medicine, Tsinghua University, Beijing 100084, China., Zhao L; Beijing National Research Center for Information Science and Technology, Institute for Precision Medicine, Tsinghua University, Beijing 100084, China. |
| Abstrakt: |
The increasing global demand for skin disease diagnostics emphasizes the urgent need for advancements in AI-assisted diagnostic technologies for dermatoscopic images. In current practical medical systems, the primary challenge is balancing lightweight models with accurate image analysis to address constraints like limited storage and computational costs. While knowledge distillation methods hold immense potential in healthcare applications, related research on multi-class skin disease tasks is scarce. To bridge this gap, our study introduces an enhanced multi-source knowledge fusion distillation framework, termed DSP-KD, which improves knowledge transfer in a dual-stage progressive distillation approach to maximize mutual information between teacher and student representations. The experimental results highlight the superior performance of our distilled ShuffleNetV2 on both the ISIC2019 dataset and our private skin disorders dataset. Compared to other state-of-the-art distillation methods using diverse knowledge sources, the DSP-KD demonstrates remarkable effectiveness with a smaller computational burden. |
