Question

the Markov chain on S = {1, 2, 3} with transition matrix

p is

0 1 0

0 1/2 1/2;

1/2 0 1/2

We will compute the limiting behavior of pn(1,1) “by hand”.

(A) Find the three eigenvalues λ1, λ2, λ3 of p. Note: some are complex.

(B) Deduce abstractly using linear algebra and part (A) that we can write

pn(1, 1) = aλn1 + bλn2 + cλn3
for some constants a, b, c. Don’t find these constants yet.

(C) The first few values of pn (1, 1) are easy to write down: they are p0 (1, 1) = 1,p1(1,1) = 0, and p2(1,1) = 0. Use these and part (B) to find an explicit formula for pn(1, 1).

(D) What is the limiting behavior of pn(1, 1) as n → ∞?

Answer 1

We need to find the probability of going from state 3 to 0 without going to state 4

The markov chain can follow the sequence - 3 2 1 0 which takes the probability 1/2*1.2*1/2

it can also return from state 2 to 3 and then repeat the process.

Similarly for state 2 and 1

Thus the total probability should be given by

1/2(1/2+1/2^2+1/2^3+ 1/2^4+............) (1/2+1/2^2+1/2^3+ 1/2^4+............)

= 1/2

Hence the answer is 1/2