Learning generalizable representations through compression

Fang, Zeming
Thumbnail Image
Other Contributors
Fajen, Brett
Sun, Ron
Zhang, Ru-Yuan
Sims, Chris
Issue Date
Cognitive science
Terms of Use
This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute (RPI), Troy, NY. Copyright of original work retained by author.
Full Citation
The ability of humans and other animals to generalize from their past experiences to novel situations is at the heart of intelligent behavior. Generalization is more likely to occur between objects that share some similarities, either perceptually or functionally. The functional- similarity-based generalization is often summarized as acquired equivalence (AE). AE has been ubiquitously documented in humans and animals, but its nature is still not well understood. Here, we propose an interpretation of the AE phenomenon by postulating that this generalization actually reflects the processes of representation compression. We formalize a representation compression framework on the basis of the rate-distortion theory (RD), a branch of information theory that characterizes a fundamental tradeoff between the complexity and accuracy of information processing. A reinforcement learning model derived from the RD theory was able to replicate human functional-similarity-based generalization. The model worked reasonably well in capturing human learning and generalization behaviors, even in an extended AE experiment paradigm where perceptual (visual) similarity was incorporated. We also identified from the model a set of low-level cognitive mechanisms (categorization and selective attention) proposed in the current AE theories to underlie the generalization in the AE task. We conclude that the representation compression framework provides a unified explanation of human AE.
December 2022
School of Humanities, Arts, and Social Sciences
Dept. of Cognitive Science
Rensselaer Polytechnic Institute, Troy, NY
Rensselaer Theses and Dissertations Online Collection
Restricted to current Rensselaer faculty, staff and students in accordance with the Rensselaer Standard license. Access inquiries may be directed to the Rensselaer Libraries.