Question

• Consider the state space {rain, no rain} to describe the weather each day. Let {Xn} be the stochastic process with this state space describing the weather on the nth day. Suppose the weather obeys the following rules:

  • – If it has rained for two days straight, there is a 20% chance that the next day will be rainy.

  • – If today is rainy but yesterday was not, there is a 50% chance that tomorrow will be rainy.

  • – If yesterday was rainy but today is not, there is a 70% chance that tomorrow will be rainy.

  • – If it has not rained for two days straight, there is a 90% chance that the next day will be rainy.

  1. (a) Is {Xn} a Markov chain? Why or why not?

  2. (b) Now let {Yn} be a different stochastic process defined as Yn = (Xn, Xn−1), so that Yn records the weather for the nth day as well as the (n−1)st day. What is the state space for {Yn}?

  3. (c) Show that {Yn} is a Markov chain and find its transition matrix.

Answer 1

a)

b)

Yn can {RR,RD,DR,DD}
where R- Rainy, D- Dry or no rain

c)