Preserving Quality of Information by using Semantic Relationships

Abstract

We show how semantic relationships that exist within an information-rich source can be exploited for achieving parsimonious communication between a pair of semantically-aware nodes that preserves quality of information. We extend the source coding theorem of classical information theory to encompass semantics in the source and show that by utilizing semantic relations between source symbols, higher rate of lossless compression may be achieved compared to traditional syntactic compression methods. We define the capacity of a semantic source as the mutual information between its models and syntactic messages, and show that it equals the average semantic entropy of its messages. We further show the duality of semantic redundancy and semantic ambiguity in compressing semantic data, and establish the semantic capacity of a source as the lower bound on semantic compression. Finally, we give a practical semantic compression algorithm that exploits the graph structure of a shared knowledge base to facilitate semantic communication between a pair of nodes.