Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes
Loading...
Authors
Chari, Shruthi
Acharya, Prasant
Gruen, Daniel M.
Zhang, Olivia
Eyigoz, Elif K.
Ghalwash, Mohamed
Seneviratne, Oshani
Saiz, Fernando Suarez
Meyer, Pablo
Chakraborty, Prithwish
Issue Date
2023-02
Type
Article
Language
en_US
Keywords
User-driven , Clinical explainability , Contextual explanations , Question-answering approach , Type-2 diabetes comorbidity risk prediction
Alternative Title
Abstract
Medical experts may use Artificial Intelligence (AI) systems with greater trust if these are supported by ‘contextual explanations’ that let the practitioner connect system inferences to their context of use. However, their importance in improving model usage and understanding has not been extensively studied. Hence, we consider a comorbidity risk prediction scenario and focus on contexts regarding the patients’ clinical state, AI predictions about their risk of complications, and algorithmic explanations supporting the predictions. We explore how relevant information for such dimensions can be extracted from Medical guidelines to answer typical questions from clinical practitioners. We identify this as a question answering (QA) task and employ several state-of-the-art Large Language Models (LLM) to present contexts around risk prediction model inferences and evaluate their acceptability. Finally, we study the benefits of contextual explanations by building an end-to-end AI pipeline including data cohorting, AI risk modeling, post-hoc model explanations, and prototyped a visual dashboard to present the combined insights from different context dimensions and data sources, while predicting and identifying the drivers of risk of Chronic Kidney Disease (CKD) - a common type-2 diabetes (T2DM) comorbidity. All of these steps were performed in deep engagement with medical experts, including a final evaluation of the dashboard results by an expert medical panel. We show that LLMs, in particular BERT and SciBERT, can be readily deployed to extract some relevant explanations to support clinical usage. To understand the value-add of the contextual explanations, the expert panel evaluated these regarding actionable insights in the relevant clinical setting. Overall, our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case. Our findings can help improve clinicians’ usage of AI models.
Description
Full Citation
Chari, S., Acharya, P., Gruen, D.M., Zhang, O., Eyigoz, E.K., Ghalwash, M., Seneviratne, O., Saiz, F.S., Meyer, P., Chakraborty, P. and McGuinness, D.L., 2023. Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes. Artificial Intelligence in Medicine, p.102498. Vancouver
Publisher
Elsevier
Terms of Use
Journal
Volume
Issue
PubMed ID
DOI
ISSN
0933-3657