Semantically enabled medical image understanding
Author
Agu, Nkechinyere NnekaOther Contributors
Hendler, James A.; Yan, Pingkun; McGuinness, Deborah L.; Zaki, Mohammed J., 1971-; Moradi, Mehdi;Date Issued
2022-08Subject
Computer scienceDegree
PhD;Terms of Use
This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute (RPI), Troy, NY. Copyright of original work retained by author.; Attribution-NonCommercial-NoDerivs 3.0 United StatesMetadata
Show full item recordAbstract
Medical imaging examination is the most common form of routine medical analysis, which involves several stages of reasoning and careful analysis to reach a final decision by the radiologist. Most deep learning frameworks utilize only the imaging data in its entirety to reach a clinical decision, ignoring important patient information that might be vital for the diagnosis. This includes symptoms, age, and patient history information. Furthermore, Most deep learning systems ignore domain based information and commonsense reasoning when they make their prediction. In this work, we designed and implemented an AI system that can use both the anatomical information and patient history information while making decisions. In particular, we focused on using semantic web technologies, ontologies and knowledge graphs, to bridge the gap in existing deep learning systems. We developed a deep learning system that can detect chest x-ray pathologies within the correct anatomical location. We developed a Medical Imaging and Diagnostic Ontology, MIDO,to provide a standardized format for modelling medical imaging tasks, including disease classification and disease localization. We then generated knowledge graphs from the chest x-ray images using the ontology for modelling the entities are relationships. Furthermore, we demonstrated the possibility of semantic reasoning to enhance transparency, trust, interpretability and explainability of the deep learning system through the application of the knowledge graph to increase the physician's trust in the individual patient's deep learning prediction by determining the likelihood of the system being correct, enriching the knowledge graph with additional domain based knowledge and reasoning causal relationships between different diseases. Not only that, but we conducted an in-depth evaluation of the various parts of our workflow and compared them to existing state-of-the art whenever possible.;Description
August 2022; School of ScienceDepartment
Dept. of Computer Science;Publisher
Rensselaer Polytechnic Institute, Troy, NYRelationships
Rensselaer Theses and Dissertations Online Collection;Access
CC BY-NC-ND. Users may download and share copies with attribution in accordance with a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 license. No commercial use or derivatives are permitted without the explicit approval of the author.;Collections
Except where otherwise noted, this item's license is described as CC BY-NC-ND. Users may download and share copies with attribution in accordance with a Creative Commons
Attribution-Noncommercial-No Derivative Works 3.0 license. No commercial use or derivatives
are permitted without the explicit approval of the author.