Measuring synergy using entropy

Synergy is phenomenon found everywhere in nature at all levels of organization in physics, chemistry, biology, social sciences, and the arts. The English Wikipedia defines Synergy as "the creation of a whole that is greater than the simple sum of its parts. The term synergy comes from the Attic Greek word συνεργία synergia from synergos, συνεργός, meaning working together". Synergy is an important concept that is difficult to define precisely and even more difficult to quantify. Entropy on the other hand has been used as a way to measure order or complexity. Here we explore the changes in entropy induced by diverse process described as synergistic. The paper uses seven examples reported in the literature to quantify synergy in different settings. The result shows that synergistic processes are associated to large decreases in entropy and slightly less large increases in work output of the system where they occur. Systems which suffer changes in entropy that are not associated to synergistic processes show a different pattern These results open the door to use entropy measures to identify and classify synergistic processes.


Introduction
No unambiguous definition of synergy exists, but most non-additive phenomena might qualify as synergy.Wikipedia defines it as the creation of a whole that is greater than the simple sum of its parts.That is, 1+1 > 2. This phenomena seems to be quite common everywhere (Corning 1983, Jaffe 2015) but a precise way of tackling the study of this phenomena has jet to be developed.For example, the transformation of oxygen and hydrogen atoms to for a molecule of water has often been presented as a synergistic process.Certainly, the properties of water cannot be deduced from those of oxygen and hydrogen atoms: they differ radically.It seems a clear cut case of 1+1 <> 2. The advent of quantum chemistry has allowed us to understand this phenomena in great detail, making the reference to synergy unnecessary.Thus, the presence of synergy in natural phenomena might just reveal our ignorance about them.
In thermodynamic, chemical reactions have properties that relate to its complexity.Quantitative measurements of Helmholtz free energy and Gibbs free energy or enthalpy have shown to be very useful in understanding chemical reactions.A similar approach should help us understanding synergy in complex systems.This can be done my measuring changes in the complexity of the system triggered by a given synergistic process.One way of estimating complexity is by measuring or estimating the entropy of the system.
The great advantage of science over other heuristic protocols is the use of empirical methods (Jaffe 2010).Here we want to explore examples where an empirical and quantitative approach to the study of synergy is possible.In all of them, the concept of Shannon's entropy is fundamental in understanding the emergence of synergy.In information theory, Shannon entropy is the expected value (average) of the information contained in each message or structure that contains information.The entropy of the message is its amount of uncertainty; it increases when the message is closer to random, and decreases when it is less than random.Named after Boltzmann's H-theorem, Shannon defined the entropy Η of a discrete random variable x as: where b is the base of the logarithm used, and P a probability function of states.
H is often related to the complexity of a system.The lower H the more complex structure the system possesses.The assessment of complexity depends on how it is regarded.If complexity C is regarded as the effort needed to make a description of the system, then it can be measured as Shannon's (1951) information entropy H. Since a description's entropy depends on the suitability of the language or metric used for that specific description, Kolmogorov (1965) stated that complexity equals the information needed assuming the language or metric used is the best suited for the description.Not being able to ensure we are using the best description, we have to be content with estimating Kolmogorov's complexity KC by using Shannon's information entropy.Thus KC ≈ H.
Complexity can also be regarded as the effort needed to describe the internal activity the system has to perform in order to keep itself structured and under control, and at least temporarily achieve the objective of limiting the natural and inexorable tendency of its entropy to increase.A way to asses this estimation of complexity is to contrast the Kolmogorov's complexity KC of the system with the 'distance' the system places itself from total disorder, which is 1-KC.

Methods
As H can be calculated or estimated quantitatively, here we relate measures of H to seven processes where synergy is known to be acting and where the relevant data for calculating H is available.We compare H in two states, before (H1) and after (H2 ) the synergistic process using a ratio Rho (P), and estimate the rate of change from state 1 to state 2. This unit free scalar was called the Rho of H and calculated as Ρ(H) = H2/H1.A mirror scalar called P(N) was the unit free quantity transformed to a scalar related to negentropy so that P(N) = 1/P(H).
Estimates of work output or efficiency of a parameter that relates to the quantity or quality of the output were calculated for each example.These values were used to calculate the ratio Rho by dividing the values associated with the lowest entropy in the final state after the synergistic process occurred by those associated with the highest entropy before the synergistic process started.We called this ratio P(W).
A summary of the data is presented in Table 6.Jaffe & Hebling-Beraldo 1993) Living in society must have benefits for the inhabitants if societies are to be sustainable.That is, social synergy allows social organisms to be more efficient in resource use than non-social ones.Data measuring the efficiency of energy consumption in ant societies of different social complexity and size are available.The study used here finds that ant species with higher polymorphisms (i.e.larger number of morphological distinct worker castes) have a lower per capita energy requirement for workers.Specifically, among ants in the family Formicinae species with 1 worker caste consume on average about 1.1 more energy than ant species with 2 worker castes.Among Myrmicinae the relationship is 1.7 times, and among Attini (leaf-cutter ants) with Atta species having >2 castes, the proportion is at least 2. This gradation corresponds well with the difference in social complexity of monomorphic and polymorphic ant societies among these three groups of ants.If P(H) is calculated just using the number of castes (1 and 2), a great underestimation of real complexity of these ant societies, we have that Shannon entropy of the society decreases at least by half.That is, P(N) = 2 ; whereas the Rho P(W) for work output, as estimated for Myrmicinae's increase in energy efficiency with increased number of castes was 1.7.

Size of social aggregates and social synergy (Data taken from Jaffe 2010)
Larger cities are more efficient in their use of energy than smaller ones.The exponent by which per capita consumption of electricity diminishes according to the number of inhabitants a city hosts varies from -0.47 in Brazil of -0.18 in Denmark.If we accept that heterogeneity in quality of services in Brazil is much higher than in Denmark, and that smaller rural towns have less services than larger metropolis in Brazil, we can assume the proportionality exponents is related to the difference in complexity of large and small cities.That is, in Brazil cities are 2.6 times more heterogeneous regarding complexity of services than those in Denmark.Among ant societies, these difference is at least 4.6 times.Here we assume that this difference is a proxy of 1/H.Jaffe et al. 2013a,b andJaffe, Rios, Florez 2013a).

Science, Economics and Social Synergy (Data taken from
The complexity of an economy and of a scientific community can be calculated with great accuracy.But both have to be calculated differently.Economic Complexity (Hausmann 2011) Table 1 shows the relationship between different ways of calculating complexity of the knowledge of a society with Economic Complexity (ECI) as calculated by Hausmann et al (2011) and Economic Growth as calculated by the World Bank as Gross Domestic Product per capita (GDPc).The knowledge of a society can be calculated as ECI, as academic publications per capita and as a scientific complexity index (SCI) calculated in the sane way as ECI but using academic publications as output rather than exports of products as in ECI (Jaffe et al 2013a).Another index that showed to be related to the sophistication of a scientific community is the Revealed Comparative Advantages or RCA (Laursen 1998).Specifically RCS in Neuroscience (Jaffe et al 2013b).  1 shows that the index for knowledge complexity that best correlate with economic growth is Publications/capita.The log of this value as reported in the mentioned paper ranged between -2.5 and 0.6 (P(N) = 3.1), whereas the log of GDPc ranged between 2.4 and 5.0 (P(W) = 2.6).That is the ratio was >2 in both cases.We can thus relate P(W) estimated as the ratio of the extremes in GDP, and P(H) as the ratio of the extremes of scientific complexity or complexity of knowledge managed by the society.

Division of labor and the invisible hand (Jaffe 2015).
In simulations of economic markets exploited by societies of agents using Sociodynamica, the amount of resources a society without division of labor could accumulate was 8 units, compared to 63 units accumulated by societies with division of labor into 3 discrete categories (miners, farmers and traders).Thus, a P(N)= 3 achieved economic gains of 7.8 times which makes P (W) = log(63)/log(8) = 2.96.This result was used as a clear demonstration that division of labor allows for synergistic interactions of agents which allow economic markets to work efficiently, triggering economic growth.This example is considered as a classical example of the working of synergy.(Jaffe & Perez 1989).

Brain complexity and social synergy
Brain complexity might be thought is an adaptive feature to allow for complex cognitive task in their bearers.If more complex societies require more complex tasks from their participating individuals, we would expect a relationship between bran complexity and complexity of the society the individuals form.Even a synergistic relationship might exist.Data from ant societies however reveal a different result (Table 2) In some cases, as the overall surface of a brain structure called the Calyx, This relationship is concave with a maximum value in species with an intermediate level of social complexity such as Acromyrmex octospinosus.In the most socially complex society studied, Atta laevigata, individuals showed a lower index of overall surface of the calyx.This relationship might suggest that socially very complex societies require cognitive less developed individuals.
The asymmetry between the two calyxes, however, was related positively and continually with social complexity, This parameters is related to the complexity of the brain rather than to its total capacity.We choose this last index to make P(N) estimates.The rate P(N) between the minimum and maximum value of this index was 16 or in entropy scalars log(16) = 4 Estimates of work output P(W) were two.The degree of worker caste polymorphism, and the estimated size of the average colony in a log scale.Each of these two different estimates of social complexity are related to the complexity of the information content the society has to manage.The ratio P (W) between maximum and minimum vales for each of them was 2.7 and 3 respectively 3.6.Synergies and language (Febres, Jaffe, Gershenson 2014 andFebres &Jaffe 2015).
Languages can be regarded as systems formed by large number of symbols and rules.For natural languages, even though there are other options, words can play the role of symbols.As a message written in a natural language is more organized, the entropy of the message reduces making it prone for a better understanding and therefore more effective as an information transmission process.We measured the entropy based on the word frequency of speeches written in English and Spanish.Assuming that those writers who won the literature Nobel Prize are masters of the use of natural languages, we compared the entropy computed for their speeches with the entropy computed for the speeches of non-Nobel writers.We also considered the readability index RES (Flesch 1951) of English texts as possible reflex of synergy.For Spanish texts we evaluated the readability using the perspicuity index IPSZ (Szigriszt-Pazos 1993).The result of this figures is shown in Table 3.There is a clear difference between the entropy of the Nobel laureate speeches and the non-Nobel speeches.This difference suggests that the more effective structure reached by master writers can be recognized by the ratio of entropies.This behavior is consistent for Spanish and English.In English, the P(N) is about 1.06 and in Spanish about 1.02 On the other hand, the change of readability index between Nobel writers and non-Nobel does not follow the same direction in Spanish and English; while in Spanish the presumably better use of the written language performed by Nobel laureates could be associated with a higher IPSZ, P(W) = 1.05.The same change does not show for RES index in English which showed a P(W) based on RES of 0.93 3.7.Music and entropy (Febres & Jaffé 2016).
Why we are delighted by music remains a mystery and an old subject for discussion.Perhaps it is due to the sound patterns we find in it, that we like music.If so, then the degree of understandingpredictability-of those patterns should be a source of pleasure and therefore the organization of the information perceived as music should be related to the degree of success of any musical piece.Obviously there are factors as personal experience and culture which have an important influence over the process of interpreting a sequence of sounds.We may or may not like some specific type of music; but this only supports our belief that pleasure, especially in the case of music, comes after certain degree of predictability of the sounds.This is not to oversimplify the phenomenon of pleasure and its causes; in fact sometimes pleasure comes after the surprise of hearing a different path of sounds and contrasting it with the path we expected before actually listening the piece.But in any case, we think the popularity of a music piece results from the way the composer organized the sound patterns.For a large number of symbols, -sounds is this case-more understandable patterns are those which the sounds are arranged to lower the entropy of the whole set of symbols.For the music most people like, there is a relatively low entropy and therefore low information structure that we can recall and also build a clearer map of expectations for the upcoming sounds.We have selected some of the most recognized music pieces from the Baroque and Classical periods.We compare the entropy of these pieces with the averages of the entropy for many music pieces in the same periods.Results are presented in Table 4, showing that the value of P(N) in this case is 1.04 H changed according to the estimated number of instruments used for recordings.When comparing the minimum (a single instrument) with the maximum (orchestra) we find a tendency for the larger number of instruments to cause an entropy reduction, suggesting an increase of structure for the musical pieces interpreted by a larger number of instruments.We think this might be the result of the proper use of the additional degrees of freedom introduced by the addition of instruments.We shall highlight the fact that adding instruments increases the difficulty of keeping harmonies and synchronization in polyphonic music, approaching a point in which the risk of losing structure and beauty of the sound, is an important factor in establishing the limits of the size of the orchestra.Considering the two cases here analyzed we estimate a P(N) ranging from 1.02 to 1.20 (average 1.1)

Conclusions
In some of the cases examined here, a synergistic increase in work was highly correlated to a decrease in information entropy of the system.This increase in negentropy was correlated to the efficiency of work output of the system which also increased strongly, but slightly less than that of negentropy.This is consistent with our understanding of the second law of thermodynamics, suggesting an increasing loss of information in the process of transforming it to work.
In some examples, the phenomena showing synergy was not present.That is, for languages such as English, Spanish and music, the impact of information content on the output was a reduction of entropy making the system's structure more orderly, creating the conditions that achieve higher aesthetic efficiencies.This process, however seems to be very different from that observed in known synergistic phenomena.

Table 1 .
Correlation coefficients between scientific publications / capita; Science Complexity Index; Economic Complexity Index and Revealed Comparative Advantages of Neurociences with the Economic Complexity Index and Gross National Product per capita of nations in 2010

Table 2 .
Various indicators of brain complexity for workers of different ant species showing a variety of social complexity

Table 3 .
Entropy H, Spanish perspicuity index IPSZ and English Readability index RES for groups of speeches written by Nobel laureates and non-Nobel writers.

Table 4 .
Comparison of entropy for MIDI music of Baroque and Classical periods and some of the most recognized pieces in the same period.

Table 5 .
Comparison of entropy for two pieces of MIDI music interpreted by different number of instruments.

Table 6 .
Impact of synergy estimated for several type of systems.