Let F be a subset of S. The hitting time ? _F is the first time the chain lands in or hits F. ? _F is defined by

if X _n ? F for some n > 0, and by ? _F = ? if X _n ? F for all n > 0. If F = { j} we denote the hitting time by ? _j. Also define f _ij ⁿ := P _i( ? _j = n), the probability of visiting j for the first positive time on the n ^th step having started in i. Since ? _j > 0 by definition it follows that f _ij ⁰ = 0 for all j, even j = i. Lastly let f _ij := ? _n=1 ^? f _ij ⁿ denote the probability of ever hitting j from i.

We now give a version of the Markov property for hitting times. This will be extended to the strong Markov property in Theorem 5.58. To state this result it is best to define precisely what we niean by the past before some stopping time.

Let ? be a stopping time relative to the sequence of ?-algebras . An event A belongs to the past before ?, which...