Learning generalizable representations through compression

Abstract

The ability of humans and other animals to generalize from their past experiences to novel situations is at the heart of intelligent behavior. Generalization is more likely to occur between objects that share some similarities, either perceptually or functionally. The functional- similarity-based generalization is often summarized as acquired equivalence (AE). AE has been ubiquitously documented in humans and animals, but its nature is still not well understood. Here, we propose an interpretation of the AE phenomenon by postulating that this generalization actually reflects the processes of representation compression. We formalize a representation compression framework on the basis of the rate-distortion theory (RD), a branch of information theory that characterizes a fundamental tradeoff between the complexity and accuracy of information processing. A reinforcement learning model derived from the RD theory was able to replicate human functional-similarity-based generalization. The model worked reasonably well in capturing human learning and generalization behaviors, even in an extended AE experiment paradigm where perceptual (visual) similarity was incorporated. We also identified from the model a set of low-level cognitive mechanisms (categorization and selective attention) proposed in the current AE theories to underlie the generalization in the AE task. We conclude that the representation compression framework provides a unified explanation of human AE.