Sharp Null Hypothesis Testing and Jeffreys-Lindley Paradox

Introduction

Frequentist and Bayesian hypothesis testing frameworks are used extensively in empirical research for drawing scientific conclusions. However there are some instances where confusions arise e.g., Jeffreys-Lindley paradox is a case where the two frameworks contradict with each other. This has caused confusions among data analysts for selecting a methodology for their statistical inferences. Though the paradox goes back to 1950's there hasn't been a satisfactory resolution to it so far, especially for the empirical researcher.

Method

We show that the paradox arises mainly due to the fact that, in the frequentist approach, it is allowed to have type-I errors and difference between hypothesized parameter value and its observed estimate is assessed in terms of standard error of the estimate, no matter what actual numerical difference between them is and how small the standard error is, whereas in the Bayesian methodology this has no effect due to its definition of Bayes factors. In fact, the paradox is an instance of conflict between statistical and practical significance. This can be seen as a result of using sharp null hypothesis to approximate acceptable small range of values for the parameter. We also show how that the frequentist null hypothesis testing should be modified so that its conflicting conclusions with the Bayesian method can be avoided. We also show why and how any uncertainty in p-values can be addressed.

Results

We have shown how to resolve the Jeffreys-Lindley paradox through a mathematical analysis. And it is also shown how to perform sharp null hypothesis tests so that undesirable rejections of null hypotheses are avoided.

Conclusion

It is possible to give a mathematical explanation to the Jeffreys-Lindley paradox, thus resolving it, and modify the frequentist null hypothesis testing methodology so that it has no conflict with the Bayesian hypothesis testing.