Cross-modal instance grounding for intelligent agent systems
Loading...
Authors
Wood, Peter D.
Issue Date
2019-12
Type
Electronic thesis
Thesis
Thesis
Language
ENG
Keywords
Computer science
Alternative Title
Abstract
Two modules for the OntoAgent cognitive architecture are presented. The first is a visual analyzer, which accomplishes a high-level semantic understanding of the output of computer vision or a simulation thereof. The design of the visual analyzer closely parallels the design of OntoAgent's existing textual analyzer. It infers event instances by comparing snapshots of its environment and using a new knowledge resource known as a "visual lexicon" to determine the meaning of differences it sees. The second module is a reasoner that synthesizes input from the visual analyzer and the already-existing textual analyzer using a variety of knowledge-based heuristics in order to determine coreference between object or event instances from the two modalities.
Description
December 2019
School of Science
School of Science
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY