Question

A die is thrown repeatedly until it lands ‘6’ three times in a row. Model this by a Markov chain.

Answer 1

Model the problem by Markov chain, We will have 4 states: 0,1,2 and 3.

Defined as below

State 0: No six on a dice.

State 1: One six on a dice.

State 2: Two six on a dice.

State 3: Three sixes on a dice.

Also, state 3 is an absorbing state.

Now, we need to get transition probabilities.

Therefore, the transition probability from state '0' to '1' , state '1' to '2'and state '2' to '3' is 1/6.

The transition probability from state '0' to '0', state '1' to '0' and state '2' to '0' is 5/6.

The transition probability from state '3' to '3' is 1 and state '3' to any other state will be 0.

So, the transition probability matrix will be as follows: