4.2 One-sided hypothesis tests

4.2.1 Definition

A hypothesis testing situation of the type described in Section 4.1 is said to be one-sided if the set Θ of possible values of the parameter θ is the set of real numbers or a subset of it and either

or

From the Bayesian point of view, there is nothing particularly special about this situation. The interesting point is that this is one of the few situations in which classical results, and in particular the use of P-values, has a Bayesian justification.

4.2.2 P-values

This is one of the places where it helps to use the ‘tilde’ notation to emphasize which quantities are random. If where is known and the reference prior is used, then the posterior distribution of θ given is . Consider now the situation in which we wish to test versus . Then, if we observe that we have a posterior probability

Now the classical P-value (sometimes called the exact significance level) against H0 is defined as the probability, when , of observing an ‘at least as extreme’ as the actual data x and so is

For example, if we observe a value of x which is 1.5 standard deviations above then a Bayesian using the reference prior would conclude that the posterior probability of the null hypothesis is , whereas a classical statistician would report a P-value of 0.0668. Of course p1=1–p0=1–P-value, so the posterior odds are

In such a case, the prior distribution could perhaps be said to imply prior odds of 1 (but beware! – this comes from taking ), and so we get a Bayes factor of

implying that

On the other hand, the classical probabilities of Type I and Type II errors do not have any close correspondence to the probabilities of hypotheses, and to that extent the increasing tendency of classical statisticians to quote P-values rather than just the probabilities of Type I and Type II errors is to be welcomed, even though a full Bayesian analysis would be better.

A partial interpretation of the traditional use of the probability of a Type I error (sometimes called a significance level) is as follows. A result is significant at level α if and only if the P-value is less than or equal to α, and hence if and only if the posterior probability

or equivalently