The Generation and Self-organization of Meaning in the Communication of Information and Redundancy

.


Introduction
In his contribution to Shannon & Weaver's (1949) 1 The Mathematical Theory of Communication, Warren Weaver stated (at p. 27) that "[t]he concept of information developed in this theory at first seems disappointing and bizarre-disappointing because it has nothing to do with meaning …" However, the author added that Shannon's "analysis has so penetratingly cleared the air that one is now, perhaps for the first time, ready for a real theory of meaning."Claiming such a theory, Luhmann ([1984] 1995) 2 argued that meaning ("Sinn") self-organizes in terms of communications among human beings.From this perspective meaning is generated in interactions among communications as a second-order effect on top of the communications among communicators.From this perspective meaning is generated in interactions among iterative communications.Hitherto, however, Luhmann's theory has remained far from operationalization and measurement.Using Bateson's (1972, at p. 315) 3 alternative definition of information as "a difference which makes a difference" (cf.MacKay, 1969), 4 Luhmann (1984, pp.102 ff.; 1995, pp.67f.) defined information as implying a selection: a difference can only make a difference for a system of reference that selects this difference from among other possible differences.Others have also defined information with reference to a receiving system (e.g., an observer) for whom a difference can make a difference.Varela (1979, p. 266)  5 argued that the word "information" is derived from the Latin word "in-formare" and thus the semantics call for the specification of a system of reference."Information," however, is then considered a substantive concept that varies with the system of reference instead of a formal measure of the uncertainty prevailing in a distribution.Kauffman et al. (2008, at p. 28), 6 for example, defined information as "natural selection assembling the very constraints on the release of energy that then constitutes work and the propagation of organization."In summary, using Bateson's alternative definition of information, the meaning of "information" becomes dependent on the context.
Using the same word ("information") for different concepts has led to considerable confusion.This semantic confusion can be avoided by using the words "uncertainty" or "probabilistic entropy" when Shannon-type information is meant.In our opinion, the advantage of measuring uncertainty-and redundancies, as we shall argue-in bits of information cannot be underestimated, since the operationalization and the measurement provide avenues to hypothesis testing and thus control of the theorizing (Theil, 1972). 7Whereas Shannon-type information cannot be specified in terms of Bateson's definition, "a difference which makes a difference" can be operationalized and measured in terms of (potentially negative) bits of information.

Weaver's (1949) suggestions
How can the processing of meaning be conceptualized by elaborating on Shannon's theory given his explicit statement that the "semantic aspects of communication are irrelevant to the engineering problem" (Shannon, 1948, at p. 3)?As a first step in the specification of the relevance of Shannon's engineering model for developing a theory of meaning, Weaver (1949, at p. 26) proposed two "minor additions" to Shannon's well-known diagram (Figure 1), as follows: "One can imagine, as an addition to the diagram, another box labeled "Semantic Receiver" interposed between the engineering receiver (which changes signals to messages) and the destination.This semantic receiver subjects the message to a second decoding, the demand on this one being that it must match the statistical semantic characteristics of the message to the statistical semantic capacities of the totality of receivers, or of that subset of receivers which constitute the audience one wishes to affect.
Similarly one can imagine another box in the diagram which, inserted between the information source and the transmitter, would be labeled "semantic noise," the box previously labeled as simply "noise" now being labeled "engineering noise."From this source is imposed into the signal the perturbations or distortions of meaning which are not intended by the source but which inescapably affect the destination.And the problem of semantic decoding must take this semantic noise into account."Since the "semantic receiver" recodes the information in the messages (received from the "engineering receiver" who only changes signals into messages) while having to assume the possibility of "semantic noise," a semantic relationship between the two new boxes can be envisaged.Given Shannon's framework, however, this relation cannot be another information transfer-since semantics are defined as external to Shannon's engineering model.Meanings, however, can be shared without observable communications since semantics are based on patterns of relations or, in other words, correlations.The correlations span a vector space in a topology different from the network space-a

SEMANTIC NOISE SEMANTIC RECEIVER
graph-of relations.For example, distances in the vector space are no longer Euclidean, but based on the angles (e.g., cosines) between the vectors.
A system of relations can be considered as a semantic domain.In other words, the sender and receiver are related in the graph of Figure 1, while they are correlated in terms of not necessarily instantiated relations in the background.The sender and receiver, for example, may share a (proto-)language.Note that this "language" is still naturalistic to the extent that one can also consider the gestures among monkeys as "languaging" (Maturana, 1978). 8The virtual structure of correlations provides a latent background that provides meaning to the information exchanges in relations.The correlations add up to a vector space that is a representation different from the network space.In other words, meaning is not added to the information generated in relations, but the same information can be considered meaningful from a systems perspective.
Weaver (1949, p. 24) further suggested taking Shannon's original diagram as a representation of "level A" which should be complemented with more levels (B and C) that represent how meaning is conveyed at level B, and how and why the received meaning can affect behavior (at level C)?In our opinion, meaning can only be conducive for behavior if it is codified and operates at the level of the social medium (that is, inter-personally); for example, as legislation.Thus, we have to look more carefully into the development and functions of codes in inter-human communication.Figure 2 provides the scheme that we propose for levels B and C. We specified above that the semantic and potentially linguistic relation between the semantic receiver and semantic noise is based on correlations among sets of relations at level A. In the vector space (level B), meanings can be shared, but cannot communicated because communication operates at level A. The use of language facilitates, supports, and potentially reinforces the options for sharing meaning.Unlike the primitive forms of language discussed above, human language enables us to explicate the meaning of the communication in messages.
Natural languages can be considered as the as yet undifferentiated and therefore common medium of communication.In other words, codes of communication are used at the symbolic level for regulating the use of language.The codes enable us to short cut the communication; for example, by paying the market price of a good instead of negotiating it using language.In our opinion, the codes of communication are thus candidates for Weaver's level C: the codes and their combinations also enable us to make the communications far more precise and efficient than is possible in natural languages.
The codes operate as expectations entertained reflexively in the communications among human beings.They open horizons of meaning that offer options.Options add to the redundancy as the complement of the information; adding options thus changes the maximum entropy-that is, the definition-of the system.How does this work?

The development of redundancies
The redundancy R is defined in information theory as the fraction of the capacity of a communication channel that is not used.In formula format: Brooks & Wiley (1986) 9 noted that in the case of an evolving system, the number of options N may increase, and therefore both the H of the system under study and the maximum entropy (Hmax) can increase (Hmax = log(N).In Figure 3a, we added green to the redundancy as part of the entropy: these are the options that were not realized by the system, but could have been realized.Kauffman (2000), for example, calls these possible realizations "adjacent," but his argument remained in the biological domain.Above this area, however, Brooks & Wiley (1986, at p. 43) added the label "impossible" as a legend (Figure 3a).In Figure 3b, we have added the label "technologically made feasible" to this latter area in order to indicate how the generation of new options (and hence redundancy) can be enhanced by cultural evolution which includes the levels B and C.An intentional system operates by adding redundancy without necessarily adding information (as we argued above).The simultaneous addition of information is likely-because of the coupling to relations-but this newly added uncertainty is not a condition for the generation of redundancy.Symbolic generalization of the codes regulates the generation of redundancies from above, whereas Shannon entropy is continuously generated in the historical process from below.
New options can be generated as mutual redundancy when two (or more) codes of communication are instantiated, as in the case of introducing a new technology in a market or when writing an evidence-based report for a government agency.In this case, one needs text that can be read using the various perspectives involved, and thus one generates redundancies deliberately (Fujigaki &  Leydesdorff, 2000). 10The redundancy (the green surfaces of Figure 3b) is generated by the recombination of redundancy flows.Redundancy flows are structured by codes in the communication.

The measurement of redundancy in two or more dimensions
We propose to specify mutual redundancy between systems in analogy to the concept of mutual information as specified in Shannon's theory, but using whole sets.Recall that mutual information between two random variables is formulated as follows: In addition to mutual information, the overlap can be considered as containing redundancy as a surplus of information.We can thus define an "excess" information value Υ12 -equivalent to H12 but with the plus sign, since we do not correct for the duplication in the case of redundancies-as follows: The mutual redundancy R12 can now be found by using Y12 instead of H12 in Eq. 2, as follows: Since T12 is necessarily positive (Theil, 1972: 59 ff.), it follows from Eq. 4 that R12 is negative and therefore cannot be anything other than the consequence of an increased redundancy.This reduction of the uncertainty is measured in bits of information, but the sign is negative.Therefore, this is not a Shannon-type information, since the latter information is necessarily positive (Krippendorff, 2009). 11or the three-dimensional case, one can define, in addition to the two-dimensional values of Υ (in Eq. 3), a three-dimensional value including the redundancy as follows: Furthermore, the mutual information in three dimensions can be derived (e.g., McGill, 1954; 12 Yeung, 2008) 13 as: It follows that the difference between Eqs. 5 and 6 (after subtraction) is: Using Υ-values instead of H-values for the joint entropies in Eq. 6, one obtains the mutual redundancy as follows: In the three-dimensional case, the mutual redundancy is thus identical to the mutual information in three dimensions.Leydesdorff & Ivanova (2014, at p. 392) 14 show that in the case of four dimensions R1234 = -T1234.The sign of the mutual redundancy alternates with the number of dimensions.This corrects for the otherwise inexplicable sign changes in the mutual information with increasing dimensionality.In other words, mutual redundancy in three or more dimensions is consistent, while mutual information itself is not, because of the sign changes with the dimensionality. 15n general, Eq. 6 can also be written as follows: the second bracket in Eq. 10 makes a negative contribution, whereas the terms in the first bracket of Eq. 10 are strictly positive.It follows (inductively) that for any given dimension n, one can formulate combinations of mutual informations corresponding to ∑ ( ) − ( , … , ) that are by definition positive (or zero in the null case of complete independence).For example (up to four dimensions) as follows: where the sums on the right-hand side are over the permutations of the indices.This relation can be extended for general n as …( ) …( ) (12)   where the last term on the right-hand side is equal to (−1) ... .Returning to the relation between and , it now follows instructively that: and the analogous relations for and follow in the same way from Eq. ( 12).More generally, in the case of more than two dimension, n > 2: The first bracketed term of Eq. 13 is necessarily negative entropy, while the configuration of the remaining mutual information relations contribute a second term which is comprised of mutual informations derived from the lower dimensional configurations.In other words, we model here the generation of redundancy on the one side versus the historical process of relating on the other, as an empirical balance.When the resulting R is negative, (evolutionary) self-organization prevails over organization (at specific moments of time) in the configuration under study, whereas a positive R indicates conversely a predominance of organization over self-organization as two different subdynamics.
Note that the resulting configuration of relations-as expressed in the right-hand term of Eq. 13no longer refers to level A of the Shannon model because this formulation implies a systems view, whereas the Shannon model focuses on local relations.(As noted, a negative value of R12 cannot be appreciated in the Shannon model.)In other words, the right-side term of Eq. 13 is composed of both interactional and correlational information, and the resulting sign and value of R thus include also the feed-forward and feedback loops between levels A and B. The one sign of R can also be associated with clockwise and the other with anti-clockwise rotation of the resulting vectors, whereas the values of the two terms in Eq. 13 measure the relative weights of the two rotations.

Summary and conclusions
We have extended Shannon's model of communication (at level A) with two levels (B and C) that change the linear model into an evolutionary one because feedback and feed-forward loops are possible among the levels.At level A, information is transmitted; at level B, information is organized and thus made meaningful in a vector-space.Reflexivity reveals that this vector space is constructed and therefore a potential subject of reconstruction: the possibility of reconstruction opens horizons of meaning (level C).Whereas the common language at level B tends to integration (into organization), the eigenvectors can be expected to span a vector space in different directions.In other words, this layer generates horizontal differentiation among the codes of communication as a top-down pressure.
Codes of communication are no longer actor-attributes, but operate on the communications among the communicators once the codes emerged in a self-organizing mode, that is, insofar as constraints on the communication are removed.The system itself has to find these resonances by varying historically.The generation of redundancy can enter the historical instantiations and under the condition of selfreinforcing loops tip the balance towards the prevalence of evolutionary self-organization over historical organization.We have shown how this trade-off can be followed by the measurement of mutual redundancy.
We argue that redundancy is a more crucial subject of study in a knowledge-based economy than information.For example, the number of options available to an innovation system for realization may be more decisive for its survival as the historically already-realized innovations (Petersen et al., in preparation). 16However, redundancies are not generated on the side of the variation, but by the selection mechanisms operating upon one another.When three or more selection mechanisms operate, auto-catalysis is an option, and options can then be generated at an increasing pace.Thus, horizontal differentiation is a necessary component of self-organization in the vertical dimension.The warp and woof of meaning generation and self-organization are not harmoniously integrated as in textiles, but differentiated and disturbing one another since operating at the same time.These dynamics lead to a fractal manifold in different directions.

Figure 2 :
Figure 2: Levels B and C added to the Shannon diagram (in red-brown and dark-blue, respectively).