TABLE OF CONTENTS The convergence of long run time averages to steady state investigated in the last section can be sharpened. For many chains, regardless of the initial distribution, the distribution after a very few transitions is approximately the stationary distribution!
By the explicit calculation of K n we see that as n ? ?
In other words, it doesn't matter whether we started in 0 or 1, after some time we are in state 0 with approximate probability b/( a + b) and in state 1 with approximate probability a/( a + b). These are precisely the steady state probabilities of the ATM chain. We note, moreover, that it doesn't take long to enter the stationary regime since the term (1 ? a ? b) n converges to 0 exponentially fast.
Things don't always work out so nicely even if a chain is irreducible. Consider a chain having kernel
Clearly, starting in state 0, after n steps we are in state 0 with probability 1 if n is even and 0 if n is odd. Similarly, starting in state 1, after n steps we are in state 1 with probability 1 if n is even and 0 if n is odd. It is clear K n 00 does not converge (but K 2 n 00 does) and this is a nuisance we will...
