5.10 Foster's Criterion for Stability

Theorems 5.21 and 5.44 appear quite general since the existence of a stationary distribution ? is a consequence of the theorem, not a hypothesis. On the other hand how can one check that a chain with a countable state space is positive recurrent? Below we review the Liapunov function technique for establishing the stability or positive recurrence of a Markov chain. We won't have space to investigate the recent developments in the fluid limit approach to stability but an interested reader can consult Dai (1996).

We say a chain X _n satisfies Foster's criterion if there exists a finite set A, a constant b < ? and a nonnegative real valued function V such that

Theorem 5.65

Let X _n be an irreducible Markov chain which satisfies Foster's criterion. Then E _i[ ? _A] ? V( i) + b ? _A( i) where ? _A is the return time to A.

Apply Dynkin's formula to the sequence Z _k = V( X _k).Consequently, for any i ? S and any n ? 0,

By the Markov property,

By hypothesis then

? _A( X _{k ?1}) = 0 for 2 ? k ? ? _A ? n. Consequently, summing from k = 1 to k = n,

which means E _i[ ? _A ? n]