Question

Joe Zilch is practicing basketball by repeatedly making attempts (shots) to put the ball in the basket. We label his first shot as random variable (RV) X1 , second shot as X2 , ..., nth shot as Xn , etc. When he takes the nth shot, he either makes a basket ( Xn =1) or misses
( Xn = 0 ). He finds that the result of any shot Xn depends on the outcome of his last two shots Xn−2 and Xn−1 as follows:
P( Xn =1 | he missed both of his last two shots) = 1/2

P(Xn =1 | he made one of his last two shots) = 2/3

P( Xn =1 | he made both of his last two shots) = 3/4
a). Show how Joe's basketball play may be modeled using a Markov chain. How many states are needed? (Hint: Define a state as the outcome of his last two shots). Draw a labeled state transition diagram or trellis describing the process.
b). Find the transition matrix P for the process.

Answer 1

a)

In a Markov chain, the transition probability to next step depends on the current state. That is, the result of any shot Xn depends on the current state. So, let the states of the Markov chain denote the outcome of his last two shots. There can be four cases based on the outcome of his last two shots.

• Xn−2 = 0 and Xn−1 = 0 (both shot missed) - Denoted as S00
• Xn−2 = 0 and Xn−1 = 1 (he made one of his last two shots) - Denoted as S10
• Xn−2 = 1 and Xn−1 = 0 (he made one of his last two shots) - Denoted as S01
• Xn−2 = 1 and Xn−1 = 1 (made both of his last two shots) - Denoted as S11

So, 4 states are needed.

The transition probability from S00 to S10 is 1/2 because P( Xn =1 | he missed both of his last two shots) = 1/2

The transition probability from S00 to S00 is 1/2 because P( Xn =0 | he missed both of his last two shots) = 1 - 1/2 = 1/2

The transition probability from S01 to S10 is 2/3 because P(Xn =1 | he made one of his last two shots) = 2/3

The transition probability from S01 to S00 is 1/3 because P(Xn =0 | he made one of his last two shots)  = 1 - 2/3 = 1/3

The transition probability from S10 to S11 is 2/3 because P(Xn =1 | he made one of his last two shots) = 2/3

The transition probability from S10 to S01 is 1/3 because P(Xn =0 | he made one of his last two shots) = 1 - 2/3 = 1/3

The transition probability from S11 to S11 is 3/4 because P( Xn =1 | he made both of his last two shots) = 3/4

The transition probability from S11 to S10 is 1/4 because P( Xn =0 | he made both of his last two shots) = 1 - 3/4 = 1/4

The state transition diagram is,

(b)

The transition matrix P for the process is 4x4 matrix shoes below,