4.1 Introduction

When we adopt the approach of probability theory as extended logic, the solution to any inference problem begins with Bayes theorem:

(4.1)

In a well-posed problem, the prior information, I, defines the hypothesis space and provides the information necessary to compute the terms in Bayes theorem.

In this chapter we will be concerned with how to encode our prior information, I, into a probability distribution to use for p( D H _i, I). Different states of knowledge correspond to different probability distributions. These probability distributions are frequently called sampling distributions, a carry-over from conventional statistics literature. Recall that in inference problems, p( D H _i, I) gives the probability of obtaining the data, D, that we actually got, under the assumption that H _i is true. Thus, p( D H _i, I) yields how likely it is that H _i is true, ^[1] and hence it is referred to as the likelihood and frequently written as ( H _i).

For example, we might have two competing hypotheses H ₁ and H ₂ that each predicts different values of some temperature, say 1 K and 4.5 K, respectively. If the measured value is 1.2 0.4 K then it is clear that H ₁ is more likely to be true. In precisely this type of situation we can use p( D H _i,