TABLE OF CONTENTS This chapter introduces the concept of Markov chains as applied to probability and statistics. It is defined in terms of probability vectors and stochastic matrices. These concepts are illustrated with several examples.
A stochastic process is a finite sequence of experiments in which each experiment has a finite number of outcomes with given probabilities. Alternately, a stochastic process is a family of random variables.
A vector
is called a probability vector if all of its components are non-negative and their sum is unity.
Determine which of the following are probability vectors.
Solution:
According to Definition 1, only z is a probability vector since all of its components are nonnegative and their sum is unity.
A stochastic matrix is a square matrix where each of its rows is a probability vector.
Determine which of the following are stochastic matrices.
Solution:
According to Definition 2, only Z is a stochastic matrix, since each of its rows is a probability vector.
If A and B are stochastic matrices, their product AB results in a stochastic matrix also.
If A is a stochastic matrix, all powers of A, that is, all A n matrices are stochastic matrices.
A stochastic process which satisfies the following two properties, in a sequence of trials whose outcomes are X 1, X 2, , X
