Consider a Bayesian problem of success probability estimation in a series of conditionally independent trials with binary outcomes. We study the asymptotic behaviour of the weighted differential entropy for posterior probability density function conditional on $x$ successes after $n$ conditionally independent trials when $n \to \infty$. Suppose that one is interested to know whether the coin is approximately fair with a high precision and for large $n$ is interested in the true frequency. In other words, the statistical decision is particularly sensitive in small neighbourhood of the particular value $\gamma=1/2$. For this aim the concept of weighted differential entropy is used. It is shown that when $x$ is a proportion of $n$ after an appropriate normalization the limiting distribution is Gaussian and the standard differential entropy of standardized RV converges to differential entropy of standard Gaussian random variable. Also, we found that the weight in suggested form does not change the asymptotic form of the Shannon and Renyi differential entropies, but changes the constants.
Previous Article in event
Next Article in event
Asymptotic behaviour of the weighted Shannon differential entropy in a Bayesian problem
Published:
13 November 2015
by MDPI
in 2nd International Electronic Conference on Entropy and Its Applications
session Information Theory
Abstract:
Keywords: differential entropy, Bayes' formula, Gaussian limit theorem