Question

1. Consider the Markov chain {Xn|n ≥ 0} associated with Gambler’s ruin with m = 3. Find the probability of ruin given X0 = i ∈ {0, 1, 2, 3}

2 Let {Xn|n ≥ 0} be a simple random walk on an undirected graph (V, E) where V = {1, 2, 3, 4, 5, 6, 7} and E = {{1, 2}, {1, 3}, {1, 6}, {2, 4}, {4, 6}, {3, 5}, {5, 7}}. Let X0 ∼ µ0 where µ0({i}) = 1 7 , i = 1, · · · , 7. Find P{X2 = 1}.

3 Consider a Markov chain {Xn|n ≥ 0} with state space S = {0, 1, · · · and transition matrix (pij ). Show that for i, j ∈ Ac , P{X1, · · · , Xn−1 ∈/ A, Xn = j|X0 = i} = q n ij (A), where Qn A = (q n ij (A)), QA = (qij (A))i,j∈Ac , qij (A) = pij if i, j ∈ Ac and = 0 otherwise.

Answer 1