TABLE OF CONTENTS We now consider the construction of a Markov chain { X n; n = 0, 1, 2, } taking values in a countable state space S along with the probability space 
on which X n is defined, given an initial distribution ? and a probability transition kernel K. As in Example 2.4 we construct a canonical sample space ? = S S S , and as in Example 2.10 we construct the subalgebra 
of all unions of atomic events of the form
where i k, k = 0, 1, 2, is some specific sequence in S. As before we define 
to be the smallest ?-algebra which contains all the 
, the state of the Markov chain at time n, is simply the n th coordinate function defined on ?.
According to Proposition 5.4 we must define the probability of these atomic events as follows:
The probability of an arbitrary event in 
is given by additivity. To check that there exists a probability P ? on 
which agrees with P n on 
, we must check the compatibility condition in the Kolmogorov Extension Theorem 2.73. We recall that a sequence of probability measures P n defined on 
satisfies the compatibility condition if P n+1( A) = P n
