MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment

MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment

16 Jan 2024 | Yequan Bie, Luyang Luo, Hao Chen
The paper "MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment" addresses the challenges of explainable artificial intelligence (XAI) in medical image analysis, particularly for skin disease diagnosis. The authors propose a multi-modal explainable framework, MICA, that aligns medical images and clinical concepts at multiple levels (image, token, and concept) to enhance interpretability and performance. MICA uses a CNN-based image encoder and a large language model (LLM)-based concept encoder to extract visual and textual features, respectively. The framework includes three alignment modules: image-level, token-level, and concept-level, which jointly learn the semantic correspondences between images and concepts. Experimental results on three skin image datasets demonstrate that MICA achieves superior performance and label efficiency while providing both visual and textual explanations. The method is designed to be interpretable, making it suitable for high-stakes applications in healthcare.The paper "MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment" addresses the challenges of explainable artificial intelligence (XAI) in medical image analysis, particularly for skin disease diagnosis. The authors propose a multi-modal explainable framework, MICA, that aligns medical images and clinical concepts at multiple levels (image, token, and concept) to enhance interpretability and performance. MICA uses a CNN-based image encoder and a large language model (LLM)-based concept encoder to extract visual and textual features, respectively. The framework includes three alignment modules: image-level, token-level, and concept-level, which jointly learn the semantic correspondences between images and concepts. Experimental results on three skin image datasets demonstrate that MICA achieves superior performance and label efficiency while providing both visual and textual explanations. The method is designed to be interpretable, making it suitable for high-stakes applications in healthcare.
Reach us at info@study.space
[slides and audio] MICA%3A Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment