• Login
    View Item 
    •   DSpace@RPI Home
    • Rensselaer Libraries
    • z_Technical Services
    • View Item
    •   DSpace@RPI Home
    • Rensselaer Libraries
    • z_Technical Services
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Knowledge-augmented deep learning and its applications

    Author
    Cui, Zijun
    View/Open
    Cui_rpi_0185E_12137.pdf (9.909Mb)
    Other Contributors
    Ji, Qiang; Wang, Meng; Chen, Tianyi; Xu, Yangyang; Talamadupula, Kartik;
    Date Issued
    2022-12
    Subject
    Computer and systems engineering
    Degree
    PhD;
    Terms of Use
    This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute (RPI), Troy, NY. Copyright of original work retained by author.;
    Metadata
    Show full item record
    URI
    https://hdl.handle.net/20.500.13015/6346
    Abstract
    Deep learning models have achieved remarkable success in many different fields over the past years thanks to advanced algorithmic techniques; great computational power provided by processors; and, most importantly, tremendous amounts of data. Though designed to mimic the behavior of human brains, existing deep models are still far from matching human learning abilities. Particularly, existing deep models are usually data hungry, fail to perform well on unseen samples, and lack of interpretability. In contrast, human beings can learn from limited observations, generalize well to novel settings, and explain well their predictions, due to their ability to extract, understand, and make use of domain knowledge. To mimic this ability, this thesis aims to identify domain knowledge and encode and integrate it into deep models for data-efficient, generalizable, and interpretable deep learning, which we refer to as \textit{knowledge-augmented deep learning}. Existing knowledge-augmented deep learning techniques face two main challenges: diverse knowledge representation formats and imperfect knowledge. Knowledge from different domains can be represented in different formats, including probabilistic relationships, symbolic rules, or mathematical equations. Domain knowledge is usually imperfect because it can be incomplete, fragmented, and ambiguous. Knowledge imperfection leads to uncertainty during inference. To address the first challenge, we propose different knowledge encoding and integration schemes to ensure that domain knowledge is efficiently and accurately encoded, and effectively integrated with data. To address imperfect knowledge, we propose to employ probabilistic models for compact and systematic encoding of the uncertainties. To evaluate the proposed knowledge encoding and integration methods, we consider four use cases. In use case 1, we show how to use a Bayesian network to encode the facial anatomy knowledge on probabilistic relationships among facial muscles and integrate it with data for facial action unit detection. In use case 2, we demonstrate how facial mechanics knowledge represented as ordinary differential equations is integrated into an encoder-decoder framework for facial action unit detection. In use case 3, we demonstrate that a prior probability as a prior model is used to encode ontological knowledge represented by symbolic rules and combined with a deep learning method for a knowledge graph completion task. Finally, in use case 4, we demonstrate how algorithmic knowledge about variational belief propagation is encoded into a message passing neural network through a custom loss function for probabilistic inference tasks on probabilistic graphical models.;
    Description
    December 2022; School of Engineering
    Department
    Dept. of Electrical, Computer, and Systems Engineering;
    Publisher
    Rensselaer Polytechnic Institute, Troy, NY
    Relationships
    Rensselaer Theses and Dissertations Online Collection;
    Access
    Restricted to current Rensselaer faculty, staff and students in accordance with the Rensselaer Standard license. Access inquiries may be directed to the Rensselaer Libraries.;
    Collections
    • z_Technical Services

    Browse

    All of DSpace@RPICommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    Login

    DSpace software copyright © 2002-2023  DuraSpace
    Contact Us | Send Feedback
    DSpace Express is a service operated by 
    Atmire NV