Learning generalizable representations through compression

Authors
Fang, Zeming
ORCID
https://orcid.org/0000-0002-8091-4413
Loading...
Thumbnail Image
Other Contributors
Fajen, Brett
Sun, Ron
Zhang, Ru-Yuan
Sims, Chris
Issue Date
2022-12
Keywords
Cognitive science
Degree
PhD
Terms of Use
This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute (RPI), Troy, NY. Copyright of original work retained by author.
Full Citation
Abstract
The ability of humans and other animals to generalize from their past experiences to novel situations is at the heart of intelligent behavior. Generalization is more likely to occur between objects that share some similarities, either perceptually or functionally. The functional- similarity-based generalization is often summarized as acquired equivalence (AE). AE has been ubiquitously documented in humans and animals, but its nature is still not well understood. Here, we propose an interpretation of the AE phenomenon by postulating that this generalization actually reflects the processes of representation compression. We formalize a representation compression framework on the basis of the rate-distortion theory (RD), a branch of information theory that characterizes a fundamental tradeoff between the complexity and accuracy of information processing. A reinforcement learning model derived from the RD theory was able to replicate human functional-similarity-based generalization. The model worked reasonably well in capturing human learning and generalization behaviors, even in an extended AE experiment paradigm where perceptual (visual) similarity was incorporated. We also identified from the model a set of low-level cognitive mechanisms (categorization and selective attention) proposed in the current AE theories to underlie the generalization in the AE task. We conclude that the representation compression framework provides a unified explanation of human AE.
Description
December 2022
School of Humanities, Arts, and Social Sciences
Department
Dept. of Cognitive Science
Publisher
Rensselaer Polytechnic Institute, Troy, NY
Relationships
Rensselaer Theses and Dissertations Online Collection
Access
Restricted to current Rensselaer faculty, staff and students in accordance with the Rensselaer Standard license. Access inquiries may be directed to the Rensselaer Libraries.
Collections