Question

Folk wisdom holds that in Ithaca in the summer it rains 1/3 of the time, but a rainy day is followed by a second one with probability 1/2. Suppose that Ithaca weather is a Markov chain. What is its transition probability?

Answer 1

Let the states of the Markov chain be rain and non-rainy. Then the stationary distribution (long term probability) is,

= (1/3, 2/3)

Given, the transition probability from rainy state to rainy state = 1/2

The transition probability from rainy state to non-rainy state = 1 - 1/2 = 1/2

Let the transition probability from non-rainy state to rainy and non-rainy state be p and 1-p respectively.

The transition probability matrix is,

Then,

=> (1/3) * (1/2) + (2/3)p = 1/3

=> 1/6 + 2p/3 = 1/3

=> 2p/3 = 1/6

=> p = 1/4

Thus, the

transition probability matrix is,