Towards improving the visual explainability of artificial intelligence in the clinical setting

Abstract Improving the visual explainability of medical artificial intelligence (AI) is fundamental to enabling reliable and transparent clinical decision-making. Medical image analysis systems are becoming increasingly prominent in the clinical setting as algorithms are learning to accurately classify diseases in various imaging modalities. Saliency heat-maps are commonly leveraged in the clinical setting and allow clinicians to visually interpret regions of an image that the model is focusing on. However, studies have shown that in certain scenarios, models do not attend to clinically significant regions of an image and perform inference using insignificant visual features. Here, we discuss the importance of focusing on visual explainability and an effective strategy that has the potential to improve a model's ability to focus more on clinically relevant regions of a given medical image using attention mechanisms.