Every finite state Markov chain has a stationary probability distribution

Question

I am trying to understand the following proof that every finite-state Markov chain has a stationary distribution. The proof is from here.

Let $P$ be the $k \times k$ (stochastic) transition probability matrix for our Markov chain. Now,

... $1$ is an eigenvalue for $P$ and therefore also for $P^t$ . Writing a $P^t$ invariant $v$ as $v = v^+ − v^−$ with $v^+ , v^− \in ( > \mathbb{R}_+ )^k$ , we obtain $P^t v^± = v^±$ because $P^t$ preserves the positive cone; if $v^+\neq 0$ take $ν = ( \sum v^+_i )^{-1} · v^+,$ otherwise normalize $v^−$.

The main thing I don't understand is

we obtain $P^t v^± = v^±$ because $P^t$ preserves the positive cone

Why is this true?

I also don't understand why $( \sum v^+_i )^{-1} · v^+$ works if $v^+ \neq 0$.

Is there any easier way to show that every finite state Markov chain has a stationary probability distribution?

Answer 1

The wording in this article is a little ambiguous. I thought of two interpretations, the first of which is incorrect. The second is correct but it doesn't explain the bit about "preservation of the positive cone".

It looks like it may be a case of mistakenly using that a mapping fixes a subset when it only preserves a subset. (I.e. $f|_X = \text{id}_X$ vs $\text{im}(f) \subset X$.)

Interpretation 1. Maybe the statement below is being claimed:

(*) Let $C$ be the positive cone. If $P^tv = v$ then for all $v^+,v^- \in C$ such that $v = v^+ - v^-$ we have $P^tv^+ = v^+$.

This is false unless $P$ is the identity matrix. Let $x \in C$, then according to (*) then $\bar v^\pm = v^\pm + x$ must satisfy $P^t\bar v^+ = \bar v^+$ as well. But linearity says then $P^tx = x$ too. So $P^t$ fixes the positive cone. This is only true if $P$ is the identity matrix because the span of the positive cone is the whole space.

Interpretation 2. Maybe instead it means to set $v^+$ to be the vector of positive entries of $v$ with 0s in place of negatives. E.g. if $v = (1,0,2,-7)$ then $v^+ = (1,0,2,0)$ and $v^- = (0,0,0,7)$. Then the claim would be:

If $P^tv = v$ then we have $P^tv^+ = v^+$ where $v^+$ is the vector of positive entries described above.

This is true, but I don't think the cited article offers any explanation as to why, and I don't know how to prove it without using Frobenius-Perron, which is maybe a harder theorem than the one we are trying to prove.

It is a trivial consequence of Frobenius-Perron in the case of an irreducible stochastic matrix, because one has either $v = v^+$ or $v = v^-$. This is because there is a stationary state $v$ (by F-P) and the eigenspace for $\lambda = 1$ is simple (also F-P). So any invariant vector is a scalar multiple of it and also has this property.

For reducible matrices the eigenspace for $\lambda = 1$ is no longer simple, so we can do things like $v = v_1 - v_2$ where $v_i$ is the stationary state for the $i$th block. Then $v^+ = v_1, v^- = v_2$. Following the suggestion in the article, one would then find a stationary distribution by normalizing just the positive part $v^+ = v_1$.

Answer 2

To your last question, as to whether there is a simpler way of proving existence of a stationary distribution for finite state Markov chains, that depends what tools you have at your disposal. Here is a nice and short consequence of a fixed point theorem:

Let a Markov chain $\mathbf{P}$ over $d$ states. The simplex $\Delta_d$ is a convex and compact subset of $\mathbb{R}^d$, which is a Euclidean vector space with usual inner product. We can look at the kernel as the following linear operator, \begin{equation} \begin{split} \mathbf{P} : \Delta_d &\to \Delta_d \\ \mu &\mapsto \mu \mathbf{P} \end{split} \end{equation} As $\|\mathbf{P}\|_2 \leq \sqrt{d} < \infty$, the operator is bounded and therefore continuous. As a consequence, we can apply Brouwer's fixed point theorem to show that $\exists \pi \in \Delta_d, \pi \mathbf{P} = \pi$.

Answer 3

we obtain $P^tv^\pm = v^\pm$ because $P^t$ preserves the positive cone

I think this line should be interpreted as '$P^t$ preserves the positive cone and is an $L^1$-isometry on the positive cone':

We have the decomposition $v = v^+ + (- v^-)$ where the vectors have disjoint supports and the first has nonnegative coordinates and the second has nonpositive coordinates.

Claim: The same properties hold for the decomposition $v = P^t v^+ + (-P^t v^-)$. (The positivity/negativity is clear since $P$ has nonnegative entries. The only nontrivial thing to prove is that the vectors have disjoint supports.)

Since $P^tv = v$, we have \begin{equation} P^t v^+ + (- P^t v^-) = v^+ + (- v^-). \end{equation} Also note that if we compute $L^1$ norms we get \begin{equation} \begin{split} \|P^t v^+\|_1 = \sum_{i} |(P^t v^+)_i| &= \sum_i \left| \sum_j (P^t)_{ij} (v^+)_j\right| \\ &= \sum_i \sum_j (P)_{ji} (v^+)_j \\ &= \sum_j (v^+)_j \sum_i (P)_{ji} \\ &= \sum_j (v^+)_j = \|v^+\|_1. \end{split} \end{equation} A similar computation shows that \begin{equation} \|-P^t v^- \|_1 = \|-v^-\|_1. \end{equation} And, in light of the disjoint supports, we have \begin{equation} \|v^+ + (-v^-)\|_1 = \|v^+\|_1 + \|(-v^-)\|_1. \end{equation} Combining these three computations, we obtain \begin{equation} \begin{split} \|P^t v^+ + (- P^t v^-)\|_1 &= \|v^+ + (-v^-)\|_1 \\ &= \|v^+\|_1 + \|-v^-\|_1 \\ &= \|P^t v^+ \|_1 + \|- P^t v^- \|_1. \end{split} \end{equation} In particular we have the triangle equality in $L^1$ norm for the sum of these vectors. This preceding fact combined with the fact that $P^t v^+$ and $-P^t v^-$ have opposite signs indeed shows that they have disjoint supports (small exercise).

From the claim it is easy to see that $P^t v^\pm = v^\pm.$

I hope I haven't written some nonsense.