Bayesian Estimation of the Entropy of the Half-Logistic Distribution Based on Type-II Censored Samples

This paper estimates the entropy of the half-logistic distribution with the scale parameter based on Type-II censored samples. The maximum likelihood estimator and the approximate confidence interval are derived for entropy. For Bayesian inferences, a hierarchical Bayesian estimation method is developed using the hierarchical structure of the gamma prior distribution which induces a noninformative prior. The random-walk Metropolis algorithm is employed to generate Markov chain Monte Carlo samples from the posterior distribution of entropy. The proposed estimation methods are compared through Monte Carlo simulations for various Type-II censoring schemes. Finally, real data are analyzed for illustration purposes.


Introduction
Shannon [1] proposed information theory to quantify information loss and introduces statistical entropy.Baratpour et al. [2] provided the entropy of a continuous probability distribution with upper record values and several bounds for this entropy by using the hazard rate function.Abo-Eleneen [3] suggested an efficient computation method for entropy in progressively Type-II censored samples.Kang et al. [4] derived estimators of the entropy of a double-exponential distribution based on multiply Type-II censored samples by using maximum likelihood estimators (MLEs) and approximate MLEs (AMLEs).Seo and Kang [5] developed estimation methods for entropy by using estimators of the shape parameter in the generalized half-logistic distribution based on Type-II censored samples.
This paper estimates the entropy of the half-logistic distribution (HLD) by using the maximum likelihood and hierarchical Bayesian methods when a sample is available from the Type-II censoring scheme.The HLD is obtained by folding the logistic distribution, which is widely employed in many fields such as biological sciences, engineering, and industrial.Balakrishnan [6] demonstrated that the HLD is applicable to life-testing studies.The cumulative distribution function (cdf) and probability density function (pdf) of the random variable X with this distribution are given by where σ is the scale parameter.The rest of this paper is organized as follows: Section 2 develops the maximum likelihood estimation method and provides a hierarchical Bayesian method to estimate the entropy of the HLD based on Type-II censored samples.Section 3 examines the validity of the proposed estimation methods through Monte Carlo simulations and real data, and Section 4 concludes.

Entropy Estimation
Let X 1:n , . . ., X n:n be the order statistics of random samples X 1 , . . ., X n from a continuous distribution with pdf f (x).In the conventional Type-II censoring scheme, r is assumed to be known in advance, and the experiment is terminated as soon as the r-th item fails (r ≤ n).Then the entropy of a continuous probability distribution based on Type-II censored samples is defined as where f 1,...,r:n (x 1:n , . . ., x r:n ) is the joint pdf of x 1:n , . . ., x r:n .Park [7] provided a single-integral representation of entropy (1) in terms of the hazard function, h(x) = f (x)/(1 − F (x)), as where f i:n (x) is the pdf of the i-th order statistic X i:n .Theorem 1.Let X 1:n , • • • , X r:n be a Type-II censored sample from the HLD.Then, the entropy of the HLD based on this sample is where Note that the entropy (2) is an increasing function of the scale parameter σ for a fixed value of r and that this entropy become negative for σ < e −c/r .
This completes the proof.

Maximum Likelihood Estimation
This subsection derives the MLE of entropy H and the corresponding approximate confidence interval by using useful properties of the MLE.
The likelihood function based on the Type-II censored sample in Theorem 1 is given by and the MLE σ can be found by maximizing the following log-likelihood function for σ: Then, by the invariance property of the MLE, the MLE of H is given by and its variance can be estimated as by using the delta method.Here Var (σ) is approximated by the inverse of the observed Fisher information for σ as , where Therefore, by the asymptotic normality of the MLE, the approximate 100(1 − ν)% confidence interval for H based on MLE Ĥ is given by where z ν/2 denotes the upper ν/2 point of the standard normal distribution.

Bayesian Estimation
In the absence of sources of informative or past data, Bayesian methods depend on the objective or noninformative priors.This subsection derives Jeffreys prior that is proportional to the square root of Fisher information, and considers a hierarchical Bayesian approach method for obtaining the Bayes estimators of σ and H.
Let θ = 1/σ.Then the likelihood function ( 3) is written as ( From the likelihood function ( 5), the corresponding negative of the second derivative is given by To obtain the expectation of (6 Therefore, the Jeffreys prior for θ is Hierarchical modeling is known to improve the robustness of resulting Bayes estimators while still incorporating prior information.Kim et al. [8] considered the inverse gamma distribution as a prior distribution for the scale parameter σ of the HLD when a sample is available from the progressively Type-II censoring scheme, which is a generalization of the conventional Type-II censoring scheme, and assumed that parameters of the inverse gamma distribution are known.Here the parameters are considered to be random variables, and then a hierarchical Bayesian estimation method is developed.
Because the prior of σ is the inverse gamma distribution, that of θ is the gamma distribution with the pdf In the gamma prior ( 7), Sun and Berger [9] derived the reference prior for α and β as where ψ (•) the trigamma function.
According to Han [10], the hyperparameters α and β should be chosen such that the gamma prior ( 7) is a decreasing function of θ.For 0 < α ≤ 1 and β > 0, and then the gamma prior ( 7) is a decreasing function of θ for 0 < α ≤ 1 and β > 0. Therefore, the prior ( 8) is specified with this supports.Then the hierarchical prior for θ is obtained as which is improper.Note that the hierarchical prior (9) has the same form as Jeffrey's prior (see the Appendix).Therefore, the hierarchical prior ( 9) is invariant under reparametrization.Under this prior, the posterior density function of θ is given by where Here, because 0 < g 1 (θ) ≤ 1, the following inequality is established: which is proportional to the pdf of the gamma distribution with parameters r and (n−r)x r:n + r i=1 x i:n .Therfore, the posterior density function (10) is proper even if the hierarchical prior ( 9) is improper.
Consider entropy H as a parameter itself.Then, with θ = e −(H−c)/r substituted into (10), the posterior density function of H is obtained as where Under the squared error loss function (SELF), Bayes estimators of θ and H are obtained by evaluating the following integrals: and However, because they do not take a closed form, the Metropolis-Hastings and random-walk Metropolis algorithms (see [11,12]) are employed to generate Markov chain Monte Carlo (MCMC) samples θ i (i = 1, . . ., N ) and H i (i = 1, . . ., N ) from posterior density functions ( 10) and ( 11), respectively.In the Metropolis-Hastings algorithm, the gamma distribution with parameters r and (n−r)x r:n + r i=1 x i:n is chosen as the proposal distribution.In the random-walk Metropolis algorithm, the normal distribution with parameters H and γVar Ĥ is chosen as the proposal distribution.Here γ is a parameter for obtaining the desired acceptance rate, and Roberts and Rosenthal [13] showed that the optimal acceptance rate is about 0.44 for one parameter.For faster convergence, the MLEs θ and Ĥ are used as starting values for repeat.
From the generated MCMC samples, Bayes estimators ( 12) and ( 13) are respectively obtained as respectively, where M is the number of burn-in samples.Here since θ = 1/σ, the Bayes estimator of σ is obtained as Chen and Shao [14] provided a simple method for constructing a 100(1 − ν)% highest probability density (HPD) credible interval based on MCMC samples.Let H (i) be the i-th smallest of H i and denote Then R i with the smallest width among all R i s is chosen as the 100(1 − ν)% HPD credible interval for H.

Application
This section assesses the performance of the proposed estimation methods and provides an example to illustrate the proposed method.

Simulation Study
Table 1 reports the results for the performance of the scale parameter σ in terms of the relative estimated risk (RER).The RER is simulated through Monte Carlo simulations.First, under various Type-II censoring schemes, samples are generated from the standard HLD, and then the MLE σ and the Bayes estimator σB are computed for each scheme.Because the estimated risk is the same as the risk function obtained from the SELF, the estimated risk for each estimator is computed by repeating this process 1, 000 times as where φ t is the true value of φ and φi (i = 1, . . ., N ) are estimates of φ.Therefore, the RER is given by RER φ = 1 1, 000  1, σB is more efficient than σ in terms of the RER.In addition, RERs for both estimators increase as r decreases for fixed n.
Figures 1 and 2 show the changes in averages of entropy estimators Ĥ and ĤB for various Type-II censoring schemes when sample sizes n = 10 and 20, which are obtained over 1, 000 replications.
In comparison to Ĥ, ĤB is closer to the true value H t .For n = 10, averages of both estimators increase when r ≤ 7, and for n = 20, they increase when r ≤ 18.
In the Bayesian context, since H is the random variable with the posterior density function (11), the 100(1 − ν)% HPD credible interval (H L , H U ) for H should meet the following condition: Therefore, averages of posterior probabilities (PPs) in HPD credible intervals are computed, along with coverage probabilities (CPs) of confidence intervals, based on Ĥ for ν = 0.05 and 0.1 through 1,000 simulations.Table 2 reports the results.The CPs are not very close to their corresponding nominal levels except in some cases, whereas the HPD credible intervals well satisfy equation (14).

Real Data
Consider the real data in Lawless [15], which represent the failure time in minutes for a specific type of electrical insulation material subjected to some continuously increasing voltage stress.The data as follows: 12.3, 21.8, 24.4,28.6, 43.2, 46.9, 70.7, 75.3, 95.5, 98.1, 138.6, 151.9.
Balakrishnan and Chan [16] verified that the HLD with the scale parameter provides a good fits to the data by using the quantile-quantile (Q-Q) plot.In the real data, all possible Type-II censoring schemes are considered in order to see how estimates of the entropy change.Table 3 shows the results.
For each scheme, σ and σB have very large values and Ĥ and ĤB decrease as r is decreases.In addition, the HPD credible intervals well satisfy equation ( 14).

Conclusions
This paper provides maximum likelihood and Bayesian methods for estimating the entropy of the HLD based on Type-II censored samples.With useful properties such as the invariance and asymptotic efficiency of the MLE, the MLE of entropy and corresponding approximate confidence interval are derived.In Bayesian inferences, a hierarchical Bayesian approach is considered.Noteworthy is that the form of the derived hierarchical prior is the same as that of Jeffrey's prior.In addition, the Bayesian estimation method based on this prior outperforms the maximum likelihood estimation method.

Table 1 .
RERs for the proposed estimators of the scale parameter σ.

Table 2 .
CPs of confidence intervals and averages of PPs in HPD credible intervals when ν = 0.05 and 0.1.