Multivariate information theory has long been problematic. Recently, the partial information decomposition (PID) of Williams and Beer has provided a promising axiomatic framework which clarifies the general structure of multivariate information. However, PID lacks the necessary measure of redundant information required to complete the framework; despite much recent research, no well-accepted measure of redundant information has emerged that is applicable to more than two sources and respects the locality of information. In this paper, we introduce a new framework based upon the axiomatic approach taken in PID but which aims to decompose multivariate information on a local or pointwise scale. It is shown that in order to identify when information from two sources is indeed the same information, one must consider decomposing the local mutual information into its two directed components, the specificity and the ambiguity. Based upon the axiomatic approach taken in PID, we decompose these two components separately resulting in two lattices - the specificity and ambiguity lattices. This Specificity and Ambiguity decomposition retains the appealing multivariate structure provided by PID, but applying this notion on a much more granular level enables the decomposition to identify when information is the same information. This last point is justified by providing an operational interpretation of redundancy in terms of Kelly gambling. Applying the decomposition to canonical examples from the PID literature demonstrates the unique ability to provide a pointwise decomposition, and the fact that the Specificity and Ambiguity decomposition possesses the much sought-after target chain rule. Finally, interpreting these results sheds light on why defining a redundancy measure for PID has proven to be so difficult - one lattice is not enough.
Previous Article in event
Next Article in event
Next Article in session
Pointwise Information Decomposition Using the Specificity and Ambiguity Lattices
Published:
20 November 2017
by MDPI
in 4th International Electronic Conference on Entropy and Its Applications
session Information and Complexity
Abstract:
Keywords: mutual information; pointwise; local; information decomposition; redundancy; synergy