Exercise 5.1

Consider a probability transition kernel K on three states: 0, 1, 2. Let the initial distribution be ? = (.2, .1, .7) and let the probability transition kernel K be

1. Compute P( X ₁ = 2 X ₀ = 1).

2. Compute P( X ₂₁ = 2 X ₂₀ = 1).

3. Compute P( X ₃ = 0, X ₅ = 2, X ₆ = 1 X ₀ = 1).

4. Compute EX ₀.

5. Compute E( X ₀ X ₀ = 1).

6. Compute P( X ₁ = 1).

7. Compute EX ₁.

8. Compute E( X ₁ X ₀ = 1).

9. Compute EX ₁ ².

10. Compute E( X ₁ ² X ₀ = 1).

11. Compute Var( X ₁ X ₀ = 1).

12. Compute P( X ₀ = 1 X ₁ = 2).

13. Calculate the stationary probability measure associated with K.

14. Calculate the mean number of transitions to return to 0.

Exercise 5.2

Consider the Markov chain X _n in Exercise 5.1. Define Y _n = ?{ X _n ? {1,2}}, that is define Y _n to be 1 if X _n is either 1 or 2 and 0 if X _n = 0. Is Y _n a Markov chain?

Exercise 5.3

Consider a probability transition kernel K(