– Entropy Only One Nonlinear Non-Shannon Inequality is Necessary for Four Variables

The region of entropic vectors ΓN has been shown to be at the core of determining fundamental limits for network coding, distributed storage, conditional independence relations, and information theory. Characterizing this region is a problem that lies at the intersection of probability theory, group theory, and convex optimization. A 2 -1 dimensional vector is said to be entropic if each of its entries can be regarded as the joint entropy of a particular subset of N discrete random variables. While the explicit characterization of the region of entropic vectors Γ ∗ N is unknown for N > 4, here we prove that only one form of nonlinear non-shannon inequality is necessary to fully characterize Γ ∗ 4. We identify this inequality in terms of a function that is the solution to an optimization problem. We also give some symmetry and convexity properties of this function which rely on the structure of the region of entropic vectors and Ingleton inequalities. This result shows that inner and outer bounds to the region of entropic vectors can be created by upper and lower bounding the function that is the answer to this optimization problem.


Introduction
The region of entropic vectors Γ * N has been shown to be a key quantity in determining fundamental limits in several contexts in network coding [1], distributed storage [2], group theory [3], and information theory [1].Γ

The Region of Entropic Vectors
Consider a set of N discrete random variables X = (X 1 , . . ., X N ), N = {1, . . ., N } with joint probability mass function p X (x).For every non-empty subset of these random variables X A := (X n | n ∈ A ), A ⊂ {1, . . ., N }, there is a Shannon entropy H(X A ) associated with it, which can be calculated from p If we stack these 2 N -1 joint entropies associated with all the non-empty subsets into a vector h = h(p X ) = (H(X A )|A ⊆ N ), h(p X ) is clearly a function of the joint distribution p X .A vector h ?∈ R 2 N −1 is said to be entropic if there exist some joint distribution p X such that h ?= h(p X ).Γ * N is then defined as the image of the set D = {p X |p X (x) ≥ 0, x p X (x) = 1}: The closure of this set Γ * N is a convex cone [1], but surprisingly little else is known about the boundaries of it for N ≥ 4.
With the convention that h ∅ = 0, entropy is sub-modular [1,5], meaning that and is also non-decreasing and non-negative, meaning that The inequalities (3) and ( 4) together are known as the polymatroidal axioms [1][5], a function satisfy them is called the rank function of a polymatroid.If in addition to obeying the polymatroidal axioms (3) and ( 4), a set function r also satisfies then it is called the rank function of a matroid on the ground set N .Since an entropic vector must obey the polymatroidal axioms, the set of all valid rank functions of polymatroids forms a natural outer bound for Γ * N and is known as the Shannon outer bound Γ N [1,5]: which held for entropies, but is not implied by the polymatroidal axioms.They called it a non-Shannon type inequality to distinguish it from inequalities implied by Γ N .In the next few years, a few authors have generated new non-Shannon type inequalities [6][7][8].Then Matúš in [4] showed that Γ * N is not a polyhedron for N ≥ 4. The proof was carried out by constructing several sequences of non-Shannon inequalities, including (8) is the same as Zhang-Yeung inequality (7) when s = 1.Additionally, the infinite sequence of inequalities was used with a curve constructed from a particular form of distributions to prove Γ Let's first introduce some basics in linear polymatroids and the Ingeton inner bound.Fix a N > N , and partition the set {1, . . ., N } into N disjoint sets T 1 , . . ., T N .Let U be a length r row vector whose elements are i.i.d.uniform over GF (q), and let G be a particular r × N deterministic matrix with elements in GF (q).Consider the N dimensional vector The subset entropies of the random variables {X i } obey A set function r(•) created in such a manner is called a linear polymatriod or a subspace rank functions.It obeys the polymatroidal axioms, and is additionally proportional to an integer valued vector, however it need not obey the cardinality constraint therefore it is not necessarily the rank function of a matroid.
Such a construction is clearly related to a representable matroid on a larger ground set [15].Indeed, the subspace rank function vector is merely formed by taking some of the elements from the 2 N -1 representable matroid rank function vector associated with G.That is, rank function vectors created via (9) are projections of rank function vectors of representable matroids.
Rank functions capable of being represented in the manner for some N , q and G, are called subspace ranks in some contexts [16][17][18], while other papers effectively define a collection of vector random variables created in this manner a subspace arrangement [19].
Define S N to be the conic hull of all subspace ranks for N subspaces.It is known that S N is an inner bound for Γ * N [16], which we call subspace inner bound.So far S N is only known for N ≤ 5 ( [18,19]).
More specifically, S 2 = Γ * 2 = Γ 2 , S 3 = Γ * 3 = Γ 3 .As with most entropy vector sets, things start to get interesting at N = 4 variables (subspaces).For N = 4, S 4 is given by the Shannon type inequalities (i.e. the polymatroidal axioms) together with six additional inequalities known as Ingleton's inequality [16,17,20] which states that for N = 4 random variables Thus, S 4 is usually called the Ingleton inner bound.We know Γ 4 is generated by 28 elemental Shannon type information inequalities [1].As for S 4 , in addition to the the 28 Shannon type information inequalities, we also need six Ingleton's inequalities (10), thus S 4 Γ 4 .
In [17] it is stated that Γ 4 is the disjoint union of S 4 and six cones {h are symmetric due to the permutation of inequalities Ingleton ij , so it sufficient to study only one of the cones.Furthermore, [17] gave the extreme rays of G ij 4 in Lemma 1 by using the following functions.For N = {1, 2, 3, 4}, with I ⊆ N and 0 ≤ t ≤ |N \I |, define is the convex hull of 15 extreme rays.They are generated by the 15 linearly independent functions Note that among the 15 extreme rays of G ij 4 , 14 extreme rays where a A ∈ R and a = [a A |A ⊆ N ].The resulting system of inequalities {a T h ≥ γ(a)| ∀a ∈ R 2 N −1 }, has each inequality linear in h, and the minimal, non-redundant, subset of these inequalities is uncountably infinite due to the non-polyhedral nature of Γ * N .Hence, while solving the program in principle provides a characterization to the region of entropic vectors, the resulting characterization with uncountably infinite cardinality is likely to be very difficult to use.
By studying the conditions on the solution to 11, in [3], the authors defined the notion of a quasi-uniform distribution and made the following connection between Γ * n and Λ n (the space of entropy vectors generated by quasi-uniform distributions).
From Theorem 1, we know finding all entropic vectors associated with quasi-uniform distribution are sufficient to characterize the entropy region, however, determining all quasi-uniform distributions is a hard combinatorial problem, while taking their conic hull and reaching a nonlinear inequality description of the resulting non-polyhedral set appears even harder, perhaps impossible.Thus new methods to simplify the optimization problem should be explored.Our main result in the next theorem shows that in order to characterize Γ * 4 , we can simplify the optimization problem (11) by utilizing extra structure of P 34 4 .
Theorem 2 (Only one non-Shannon inequality is necessary).To determine the structure of Γ * 4 , it suffices to find a single nonlinear inequality.In particular, select any h A ∈ Ingleton ij .The region P ij 4 is equivalently defined as: where h \A is the 14 dimensional vector excluding h A , 4 is a 15 dimensional polyhedral cone.Inside this cone, some of the points are entropic, some are not, that is to say, P 34 4 G 34 4 .From Lemma 1 we obtain the 15 extreme rays of , where each of these extreme rays are 15 dimensional, corresponding to the 15 joint entropy h A for A ⊂ N .The elements of these extreme rays are listed in Fig. 1.As shown in Fig. 1 with the green rows, if we project out h 123 from these 15 extreme rays, the only ray which is not entropic, f 34 , falls into the conic hull of the other 14 entropic extreme rays, that is to say, The p equival single n g up : π g up (h The lists of non-Shannon inequalities make the list of li ), the problem of determining whether or not [h A h T \A ] T is an entropic vector in P 34  4 is equivalent to determining if h A is compatible with the specified h \A , as P 34 4 is convex.The set of such compatible h A s must be an interval [g low (h \A ), g up (h \A )] with functions defined via ( 13) and ( 14).This concludes the proof of (12).
To see why one of the two inequalities in ( 14),( 13) is just the Ingleton inequality Ingleton 34 , observe that for the case of dropping out h 123 , the only lower bound for h 123 in G 34 4 is given by Ingleton 34 0 (all other inequalities have positive coefficients for this variable in the non-redundant inequality description of G 34 4 depicted in Fig. 2).Thus, if h ∈ P 34 4 , then h ∈ G 34 4 , and Furthermore, {Ingleton 34 =0 ∩ G 34 4 } = {Ingleton 34 =0 ∩ P 34 4 } since all {Ingleton 34 = 0} rays of the outer bound G 34 4 are entropic, and there is only one ray with a non-zero Ingleton 34 , so the extreme rays of {Ingleton 34 =0 ∩ G 34 4 } are all entropic.This means that for any h \123 ∈ π \123 G 34 4 , the minimum for h 123 specified by Ingleton 34 is attainable, and hence Thus, the problem of determining Γ * 4 is equivalent to determining a single nonlinear function g up 123 (h \123 ).A parallel proof applied for other h A with a non-zero coefficient in Ingleton ij yields the remaining conclusions.4 .Note that in each column where Ingleton 34 has a non-zero coefficient, it is the only coefficient with its sign.
The p equival single n g up : π g up (h The lists of non-Shannon inequalities make the list of lin 27 From Theorem 2, we ten nonlinear inequalities (depending on which A with h A appearing in Ingleton ij is selected), any single one of which completely determines P ij 4 , and thus, with its six permutations, determine Γ * 4 .This theorem largely simplifies the optimization problem of determining Γ * 4 , in that we only need to work on maximizing or minimizing a single entropy h A given any h \A in the polyhedral cone π \h A G ij 4 , which is entirely entropic.

Properties of g up
A (h \A ) and g low A (h \A ) Based on the analysis in the above section, once we know any one of the ten nonlinear functions, g up 1 (h \1 ), g up 2 (h \2 ), g up 34 (h \34 ), g up 123 (h \123 ), g up 124 (h \124 ), g low 12 (h \12 ), g low 13 (h \13 ), g low 14 (h \14 ), g low 23 (h \23 ), and g low 24 (h \24 ) we know P 34 4 and hence Γ * 4 .In this section, we investigate the properties of these functions, including the properties of a single nonlinear function, as well as the relationship between different nonlinear functions.The first result is the convexity of −g up A (h \A ) and g low A (h \A ).
Lemma 2. The following functions corresponding to P 34 4 are convex: Proof: Without loss of generality, we investigate the convexity of T be any two entropic vectors in the pyramid P 34 4 .Since Γ * 4 is a convex set, P 34 4 is also convex.Thus for ∀ 0 λ 1, we have λh a + (1 − λ)h b ∈ P 34 4 .According to Theorem 2, we have Furthermore, for some h a and h b to make g up 1 tight, besides (15), the following two conditions also hold: In Figure 3, f is the one of 6 bad extreme rays(extreme rays of Γ 4 that are not entropic).The rectangle formed by connecting (0,0), (2,0), (0,2) and f is the mapping of Shannon outer bound Γ 4 onto this plane.The green line connecting a and e is the projection of Ingleton 34 onto the plane.Notice we also plot inequality (7) and ( 8 give us the same value, that is to say they are identical when fixed in this hyperplane.

Conclusions
In this paper, we proved that the problem of characterizing the region of entropic vectors was equivalent to finding a single non-linear inequality solving one of ten interchangeable optimization problems.Additionally, we investigated some symmetry and convexity based properties of the functions that are the solutions to these optimizations problem.Our future work is focused on calculating upper and lower bounds for these nonlinear functions.

Figure 2 .
Figure 2. The coefficients of the non-redundant inequalities in G 344 .Note that in each column where Ingleton 34 has a non-zero coefficient, it is the only coefficient with its sign.

Figure 3 . 1 axis h 2 f 1
Figure 3. Entropic vector hyperplane with only h 1 and h 2 coordinate not fixed ) for some values of s in the figure for the comparison between Ingleton inner bound, Shannon outer bound and non-Shannon outer bound.The red dot point c is the entropic vector of the binary distribution with only four outcomes: (0000)(0110)(1010)(1111), each of the outcomes occur with probability1  4 , and following from the convention of[21], we call it the 4 atom uniform point.Since we already know a = [1 2 V ] and e = [2 1 V ] must lie on the boundary of P 34 4 , thusg up 1 ([2 V ]) = g up 2 ([2 V ]) and g up 1 ([1 V ]) = g up 2 ([1 V ]).More generally, for any entropic vector b = [x y V ] on the boundary, we have g up 1 ([x V ]) = g up 2 ([x V ]) and g up 1 ([y V ]) = g up 2 ([y V ]).Thus we can say that when we constrain the last 13 dimension of entropic vector to V = [3 2 3 3 4 2 3 3 4 4 4 4 4], the two function g up 1 and g up 2 always r l 2 are also extreme rays of S 4 and thus entropic, which leaves f ij the only extreme ray in G ij 4 that is not entropic.It is easily verified that Γ * 4 is known as long as we know the structure of six cones Γ * 4 ∩ G ij 4 .Due to symmetry, we only need to focus on one of the six cones Γ * 4.1.Derivation of a single nonlinear functionOne way to propose the problem of characterizing the entropy region is by the following optimization problem γ(a) = min The extreme rays of G 34 4 .The top row is the ray f 34 , and all of its coefficients except in the red column (corresponding to h 123 ) are the sum of the entries in the green rows.Hence π \h 123 G 34 4 is entirely entropic.1 h 2 h 12 h 3 h 13 h 23 h 123 h 4 h 14 h 24 h 124 h 34 h 134 h 234 h 1234 \h 123 P 34 4 = π \h 123 G 34 4 .Furthermore, one can easily verify that theFigure 1.h 27same statement holds if we drop any one of the 10 joint entropies h A ∈ Ingleton 34 by summing other extreme ray rows to get all but the dropped dimension.This then implies that for h A ∈ Ingleton 34 the projected polyhedron π \h A G 34 4 (from which the dimension h A is dropped) is entirely entropic, and hence π \h A P 34 4 = π \h A G 34 4 .Hence, for some h A with a non-zero coefficient in Ingleton 34 , given any point h \A ∈ π \h A G 34 4 (= π \h A P 34 4