Towards explainable and actionable bayesian deep learning

Loading...
Thumbnail Image
Authors
Wang, Hanjing
Issue Date
2024-12
Type
Electronic thesis
Thesis
Language
en_US
Keywords
Electrical engineering
Research Projects
Organizational Units
Journal Issue
Alternative Title
Abstract
Despite significant progress in many fields, conventional deep learning models cannot effectively quantify their prediction uncertainties. They are typically overconfident in areas they do not know and they are prone to adversarial attacks and out-of-distribution inputs. To address these limitations, we propose utilizing explainable and actionable Bayesian deep learning (BDL) methods to perform accurate and efficient uncertainty quantification, identify uncertainty sources, and develop strategies to mitigate their impacts on prediction accuracy. Existing BDL methods have several shortcomings: First, they are either accurate but computationally intractable or efficient but inaccurate in uncertainty quantification. Second, they typically have limited explainability and lack an understanding of the sources of uncertainties. Finally, they often fail to mitigate uncertainties to improve model prediction performance. To address these shortcomings, this thesis focuses on three thrusts: uncertainty quantification (UQ), uncertainty attribution (UA), and uncertainty mitigation (UM). For UQ, we introduce advanced techniques to achieve a better efficiency-accuracy trade-off. The first approach enhances traditional ensemble methods by increasing component diversity, achieving state-of-the-art UQ performance. The second approach integrates an evidential neural network with Bayesian deep learning, allowing for simultaneous prediction and UQ in a single forward pass, which significantly improves computational efficiency without compromising accuracy. Additionally, we developed a gradient-based UQ method for pretrained models, enabling easy calculation of epistemic uncertainty without the need for model refinement and access to the training data. In UA, to improve explainability, we developed both gradient-based and optimization-based methods to identify problematic regions in the input that contribute to prediction uncertainty. The gradient-based method offers competitive accuracy, relaxed assumptions, and high efficiency, whereas the optimization-based method formulates UA as an optimization problem, achieving state-of-the-art performance by learning informative perturbations. Finally, for UM, we leverage insights from uncertainty attribution to develop strategies that enhance model performance. By using uncertainty attribution maps as attention mechanisms, our approach directs the model's learning toward more informative regions with low uncertainty, improving prediction accuracy and robustness.
Description
December2024
School of Engineering
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY
Terms of Use
Journal
Volume
Issue
PubMed ID
DOI
ISSN
EISSN
Collections