Question

Suppose that a production process changes states according to a Markov process whose one-step probability transition matrix is given by

0 1 2 3

0 0.3 0.5 0 0.2

1 0.5 0.2 0.2 0.1

2 0.2 0.3 0.4 0.1

3 0.1 0.2 0.4 0.3

a. What is the probability that the process will be at state 2 after the 105th transition given that it is at state 0 after the 102 nd transition?

b. What is the probability that the process will be at state 2 after the 3rd transition if P(X0 = 0) = 0.2, P(X0 = 2) = 0.3 and P(X0 = 3) = 0.5.

c. Explain why this Markov chain has a steady state distribution and determine steady state probabilities.

d. Suppose that the process is “in-control” if it is in states 0 or 1 and is “out-of-control” if it is in states 2 or 3. In the long run, what fraction of the time is the process in-control?

e. In the long run, what fraction of transitions is from an out-of-control state to an in-control state?

f. What is the long run time average “out-of-control “cost if we incur 7TL and 4TL at every time the process visits state 2 and 3, respectively?

Answer 1

Answer;

a)

we need P³₀₂

P <- matrix(c(.3,.5,0,.2,.5,.2,.2,.1,.2,.3,.4,.1,.1,.2,.4,.3),nrow=4,byrow=T)

P

P %^% 3

[,1] [,2] [,3] [,4]

[,1] 0.306 0.326 0.198 0.170

[,2] 0.324 0.306 0.206 0.164.

[,3] 0.306 0.316 0.220 0.158

[,4] 0.288 0.304 0.250 0.158

hence 0.198 is correct

b)

c(0.2, .3, .5) %*% (P %^% 3)

[,1] [,2] [,3] [,4]

[,1] 0.297 0.312 0.2306 0.1604

hence probability of being in state 2 = 0.2306

c)

steady state distribution

mu P = mu

0.3087071 0.3139842 0.2137203 0.1635884

d)

fraction of out-of-control = 0.21337203 + 0.1635884

= 0.3773087