Question

I am having problems with a probability/stochastic process question regarding discrete markov chains. I get the intuition behind the questions, but I am unsure on how to prove them. P is a finite transition matrix.

Question:
Let P have a finite state space of size N + 1. Suppose that P is irreducible. Let «i» and «j
» be a fixed and different states.
Prove that Pij⁽ⁿ⁾ > 0 for some n ≤ N.
Prove that Pii⁽ⁿ⁾ > 0 for some n ≤ N + 1.

Answer 1

The state j is said to be accessible from a state I usually denoted as i->j ,if there exist some n>=0

Pin(n) =p(Xn=j/X0=I)>0

That is one can get from the state i to the state j in n steps with probability pin(n) . If both i->j hold true then the state I and j communicate

Let I annd j be two distinct a area of Markov chain

For example

We have finite irreducible like tpm this ,

P=1/2 1/2

1/2 1/2

P00(1)= 1/2 , P00(2)=1/4, P00(3)=1/8 , P00(4)= 1/16

P11(1)= 1/2 , P11(2)= 1/4 ,P00(3)= 1/8 , P00(4)= 1/16

And

P01(1)= 1/2 , P01(2)= 1/4 ,P01(3)= 1/8 ,P01(4)=1/16

Similar for P10(n)= 1/2^i ..where n=1,2,3,4 and I =1,2,3,4

Hence we can say that for finite irreducible Markov chain

Pij(n)>0 for some n>=N

And Pii(n)>0 for some n>=N+1

E