I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix
\begin{bmatrix}1/3&2/3&0&0&0\\ 1/2&1/2&0&0&0\\ 0&1/2&0&1/2&0\\ 0&0&0&1/4&3/4\\ 0&0&0&1/3&2/3\end{bmatrix}
Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.
does anybody have an idea?
$\endgroup$ 12 Answers
$\begingroup$Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
$\endgroup$ 2 $\begingroup$You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,\frac 4 3 t,0,s, \frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $\sum x_i=1$.
$\endgroup$
