Question

This is an example problem from the book " Introduction to Probability Models (Ninth Edition) by Ross. This is example 4.10 and the question is

A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i with probability Pi, i = 1, 2, 3, 4, _4 i=1 Pi = 1. If the pensioner has more than 3 at the end of a month, he gives the amount greater than 3 to his son. If, after receiving his payment at the beginning of a month, the pensioner has a capital of 5, what is the probability that his capital is ever 1 or less at any time within the following four months.

But I need to solve an extra part of that question using this example. Take time at the beginning of each month (after the pensioner receives $2,000) and write down the transition matrix of the Markov Chain of the money the pensioner has...

I need to find out this transition matrix part but don't know how to solve it

Answer 1