Complexity as causal information integration

Complexity measures in the context of the Integrated Information Theory of consciousness, developed mainly by Tononi [7], try to asses the strength of the causal connections between different neurons. This is done by minimizing the Kullback-Leibler-Divergence between a full system and one without causal connections. Various measures have been proposed in this setting and compared in, for example, [3],[5]. Oizumi et al. develop in [6] a unified framework for these measures and postulate properties that they should fulfill. Furthermore, they introduce an important candidate of these measures, denoted by Φ, based on conditional independence statements. Unfortunately it cannot be computed analytically in general and the KL-Divergence has to be optimized numerically.

We propose an alternative approach using a latent variable which models a common exterior influence. This leads to a measure, causal information integration, that satisfies all of the required conditions provided the state space of the latent variable is large enough and it can serve as an upper bound for Φ. Our measure can be calculated using an iterative information geometric algorithm, the em-algorithm. Therefore we are able to compare its behavior to existing integrated information measures.

[1] S. Amari, N.Tsuchiya, and M. Oizumi. “Geometry of Information Integration”. In: Information Geometry and Its Applications.

[2] N. Ay. “Information Geometry on Complexity and Stochastic Interaction”. Entropy(2015).

[3] M. Kanwal, J. Grochow, and N. Ay. “Comparing Information-Theoretic Measures of Complexity in Boltzmann Machines”. Entropy(2017).

[4] C. Langer. “Theoretical Approaches to Integrated Information and Complexity”. master thesis, 2019.

[5] P. Mediano, A. Seth, and A. Barrett. “Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation”. Entropy(2019).

[6] M. Oizumi, N. Tsuchiya, and S. Amari. “Unified framework for information integration based on information geometry”. PNAS. 2016.

[7] G. Tononi. “An information integration theory of consciousness”. BMC Neuroscience(2004).