2.6 Highest density regions

2.6.1 Need for summaries of posterior information

In the case of our example on Ennerdale granophyre, all the information available after the experiment is contained in the posterior distribution. One of the best ways of conveying this information would be to sketch the posterior density (though this procedure is more difficult in cases where we have several parameters to estimate, so that θ is multi-dimensional). It is less trouble to the statistician to say simply that

although those without experience may need tables to appreciate what this assertion means.

Sometimes the probability that the parameter lies in a particular interval may be of interest. Thus, there might be geological reasons why, in the above example, we wanted to know the chance that the rocks were less than 400 million years old. If this is the case, the probability required is easily found by use of tables of the normal distribution. More commonly, there are no limits of any special interest, but it seems reasonable to specify an interval in which ‘most of the distribution’ lies. It would appear sensible to look for an interval which is such that the density at any point inside it is greater than the density at any point outside it, and it would also appear sensible to seek (for a given probability level) an interval that is as short as possible (in several dimensions, this means that it should occupy as small a volume as possible). Fortunately, it is clear that these conditions are equivalent. In most common cases, there is one such interval for each probability level.

We shall refer to such an interval as a highest (posterior) density region or an HDR. Although this terminology is used by several authors, there are other terms in use, for example Bayesian confidence interval (cf. Lindley 1965, Section 5.2) and credible interval (cf. Edwards et al. 1963, Section 5). In the particular example referred to aforementioned text, we could use the well-known fact that 95% of the area of a normal distribution lies within standard deviations of the mean to say that , that is (399, 427) is a 95% HDR for the age θ given the data.

2.6.2 Relation to classical statistics

The traditional approach, sometimes called classical statistics or sampling theory statistics would lead to similar conclusions in this case. From either standpoint

and in either case the interval (399, 427) is used at a 95% level. However, in the classical approach, it is x that is regarded as random and giving rise to a random interval which has a probability 0.95 of containing the fixed (but unknown) value θ. By contrast, the Bayesian approach regards θ as random in the sense that we have certain beliefs about its value, and think of the interval as fixed once the datum is available. Perhaps the tilde notation for random variables helps. With this, the classical approach amounts to saying that

with probability 0.95, while the Bayesian approach amounts to saying that

with probability 0.95.

Although there is a simple relationship between the conclusions that classical and Bayesian statisticians would arrive at in this case, there will be cases later on in which there is no great similarity between the conclusions arrived at.