5.9 The Watched Process

Consider an irreducible, recurrent Markov chain X _n and consider some subset A in the state space. If we watch the chain X _n only when it returns to A, we obtain the process watched on A. More precisely define ? _A( n) to be the n ^th return time to A = where ? _A(0) = 0. Let W _n = X _{? A( n)}. Take i, j ? A. Take Y = ?{ X _{? A(1)} = j} so

since the shift ? _{? A( n)} cuts off the trajectory of X before time ? _A( n). By Theorem 5.58, with ? taken to be ? _A( n),

Hence, conditioning on the event { W ₀ = i ₀, , W _{n ?1} = i _{n ?1}, W _n = i} we get

This means the process W _n is a Markov chain with a stationary transition kernel.

We next examine the transition kernel _AK for the process on A. Define

which gives the probability, starting in i, of hitting j (which may or may not be in A) on the m ^th step, having stayed in A ^c in the preceding steps. Next extend the definition...