TABLE OF CONTENTS Consider a probability transition kernel K ij on a state space S. We say a state j is accessible from state i if for some n ? 0, K ij n > 0. We say two states i and j communicate if each is accessible from the other.
The state space may be divided into disjoint sets called communication classes. States within a communication class all communicate.
Proof: Suppose k communicates with both i and j. Therefore there exist n and m such that K ik m > 0 and K kj n > 0. Now by the Chapman-Kolmogorov equation
Hence j is accessible from i. Similarly i is accessible from j, so i and j communicate. Communicating states form an equivalence class so sets of communicating states are necessarily equal or disjoint. (The notion of equivalence class is reviewed in the Appendix.)
Consider the probability kernel
Clearly K has two communication classes, the first two states and the last two.
We say a Markov chain is irreducible if there is only one communication class.
We say a positive measure ? on S is stationary if
and is a stationary probability measure if in addition ? i ?( i) = 1.
If the initial distribution of a Markov...
